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SUMMARY 


This  technical  paper  documents  a  study  to  validate  and  verify  the  Integrated 
Maintenance  Information  System  (IMIS)  diagnostic  module.  This  effort,  while  focusing  on  the 
diagnostic  module,  evaluated  limited  aspects  of  the  technical  order  authoring  and  presentation 
module.  Although  testing  was  done  on  the  IMIS  diagnostic  system,  the  scenarios  investigated 
are  applicable  to  most  diagnostic  systems,  and  the  results  and  recommendations  may  also  have 
wider  applicability. 

This  effort  consisted  of  three  segments:  scenario  development,  test  and  analysis,  and 
recommendations.  The  scenarios  described  conditions  where  pitfalls  may  exist  in  the  current 
diagnostic  system.  Test  and  analysis  focused  on  the  procedures  and  results  of  demanding 
scenarios  within  the  IMIS  diagnostic  module.  Recommendations  axe  provided  based  on 
adverse  testing  results. 

The  investigation  proved  that  most  fault  isolation,  rectification  inefficiencies,  and 
failures  were  the  result  of  inaccurate  or  incomplete  supporting  data  bases.  Diagnostic%fficicnc> 
was  compromised  as  a  result  of  intermittent  faults  and  faults  that  could  not  be  duplicated.  An 
efficient  diagnostic  model  should  provide  provisions  to  check  data  entries  and  to  handle  faults 
that  come  and  go  randomly  or  that  cannot  he  duplicated. 

The  following  recommendations  were  provided  to  resolve  problems  with  diagnostic  or 
supporting  data  base  inefficiencies: 

1 .  Incorporate  a  strategy  in  the  diagnostic  module  to  handle  faults  that  are  intermittent 
or  cannot  be  duplicated. 

2.  Precisely  correlate  faults,  symptoms,  and  repair  actions  to  prevent  unmodeled  or 
useless  supporting  data  base  information. 

3.  Develop  methods  to  allow  more  flexibility  and  ease  in  correcting  human  inputs 
during  use  of  the  system. 

4.  Incorporate  sequencing  capabilities  to  handle  cannibalization  and  facilitation  of  other 
maintenance. 
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PREFACE 


This  paper  documents  the  validation  and  verification  of  the  Integrated  Maintenance 
Information  System  (IMIS)  diagnostic  module.  Version  3.3  for  the  Air  Force  Human 
Resources  Laboratory,  Combat  Logistics  Branch  (LRC),  under  the  terms  of  Contract 
#F33615-88-C-0004,  Task  Order  #0004-02.  The  IMIS  diagnostic  module  development  is  part 
of  the  overall  IMIS  concept  that  will  demonstrate  the  capability  to  access  and  integrate 
maintenance  information  from  multiple  sources  and  present  the  information  to  technicians 
through  a  rugged,  hand-held  computer. 

Research  was  performed  by  the  Dayton  regional  office  of  Systems  Exploration,  Inc. 
(SEI).  Principal  investigators  were  Garth  Cooke,  Nicola  Maiorana,  Theodore  Myers,  and 
Johnnie  Jemigan.  The  Air  Force  technical  monitor  for  this  task  was  Captain  Dwayne  Mason. 
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I.  INTRODUCTION 


Purpose 

The  purpose  of  this  paper  is  to  document  the  validation  and  verification  of  Version  3.3 
of  the  Integrated  Maintenance  Information  System  (IMIS)  diagnostic  module.  Results  may  be 
applicable  to  assess  performance  of  other  diagnostic  modules.  Identifying  inefficiencies  and 
errors  in  algorithms,  theories,  and  supporting  data  bases  is  important  in  examining  system 
capability  and  defining  necessary  enhancements  for  the  IMIS. 


Background 


The  objective  of  IMIS  is  to  improve  the  capabilities  of  base-level  aircraft  maintenance 
organizations  by  providing  technicians  with  a  single,  integrated  information  system  for 
intermediate  and  organizational  maintenance. 


IMIS  technology  demonstrations  will  use  a  portable  computer  to  interface  with  on- 
aircraft  systems  and  ground-based  computer  systems  to  provide  a  single,  integrated  source  of 
the  information  needed  to  perform  maintenance  on  the  flightline  and  in  die  intermediate  shop 
level  (Figure  1).  The  IMIS  will  access,  integrate,  and  display  maintenance  information  for  the 
technician.  It  will  provide  the  technician  with  direct  access  to  several  maintenance  information 
systems  and  data  bases  including  historical  data  collection  and  analysis  systems,  supply  data 
bases,  automated  technical  order  systems,  and  automated  training  systems.  The  IMIS  will 
display  technical  instructions,  provide  intelligent  diagnostic  and  rectification  advice,  provide 
aircraft  battle  damage  assessment  information,  analyze  in-flight  performance  and  failure  data, 
analyze  aircraft  historical  data,  and  interrogate  on-aircraft  built-in  test  capabilities.  It  will  also 
provide  the  technician  with  easy,  efficient  methods  to  receive  work  orders,  report  maintenance 
actions,  order  parts  from  supply,  and  complete  computer-aided  training  lessons.  The  portable 
computer  will  display  all  of  the  information  the  technician  needs  for  on-equipment  maintenance 
and  diagnostics.  The  portable  computer  will  make  it  possible  to  present  quality  information  by 
taking  advantage  of  the  computer's  ability  to  tailor  information  to  a  technician’s  level  of 
expertise. 

The  diagnostic  capability  is  a  key  element  of  IMIS.  The  IMIS  diagnostic  module  was 
designed  as  a  generic  fault  isolation  and  rectification  tool  capable  of  efficient  operation  in  both 
single  fault  and  multiple  fault  environments.  The  tool  is  model  based;  hence,  it  can  operate 
equally  well  on  any  subsystem  (not  just  electronic  systems),  and  depends  only  on  development 
and  use  of  an  effective  system  model  which  accurately  defines  fault,  test,  and  rectification 
relationships. 


As  implemented  in  IMIS,  the  diagnostic  module  is  very  closely  integrated  with  the 
technical  order  Authoring  and  Presentation  System  (APS).  Procedures  for  performing  tests 
and  rectifications  and  task  times  for  performing  these  activities  are  key  elements  contained  in 
the  technical  order  data  base.  Furthermore,  all  diagnostic  outputs  to  the  user  interface  are 
controlled  by  the  presentation  system.  Hence,  this  validation  and  verification  effort  must 
evaluate  some  limited  aspects  of  the  IMIS  technical  order  presentation  module. 
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Figure  1.  Integrated  Maintenance  Information  System. 

Proper  performance  of  the  diagnostic  module  is  essential  to  the  ultimate  usefulness  of 
IMIS;  therefore,  validation  and  verification  efforts  must  thoroughly  test  the  diagnostic 
module's  capability  to  operate  in  real  world  conditions.  The  scenarios  prepared  for  this  effort 
and  the  tests  performed  to  evaluate  diagnostic  capabilities  against  these  scenarios  can  be  applied 
to  any  diagnostic  model  proposed  for  use  in  an  organizational  maintenance  environment  on  any 
kind  of  subsystem  (electronic,  mechanical,  hydraulic,  etc.). 


Scope 

The  scope  of  this  effort  was  to  validate  and  verify  the  diagnostic  efficiency  and 
accuracy  of  the  IMIS  diagnostic  module.  This  was  accomplished  through  (a)  defining 
scenarios  that  present  demanding  diagnostic  problems,  (b)  developing  and  performing  tests  that 
reflect  the  scenarios,  (c)  documenting  and  analyzing  test  results,  and  (d)  recommending 
solutions  to  shortfalls  encountered. 


Scenarios  were  developed  to  examine  three  basic  issues:  fault  processing  problems, 
data  base  inaccuracies,  and  user  interface  problems.  The  fault  processing  issues  address  real 
world  conditions  that  place  demands  on  current  diagnostic  modeling  theory  and 
implementation.  Data  base  issues  address  results  that  could  occur  with  incorrect  data  entry, 
incorrect  data  transfer,  or  improper  data  manipulation.  User  interface  issues  address  the 
difficulties  that  a  user  may  experience  when  correcting  mistaken  entries  and  the  results  of  an 
erroneous  user  input  on  the  effectiveness  and  the  efficiency  of  the  diagnostic  sequence. 
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II.  SCENARIOS 


The  following  scenarios  describe  conditions  that  could  tax  diagnostic  effectiveness  but 
should  be  handled  by  a  diagnostic  system.  These  scenarios  provided  for  validation  and 
verification  of  diagnostic  module  capabilities,  regulated  and  guided  test  development  and 
documentation  of  results,  and  provided  a  framework  for  developing  recommendations  that 
stemmed  from  test  results. 


Fault  Processing  Issues 


Can  Not  Duplicate  (CND) 

A  CND  problem  occurs  when  the  maintenance  technician  cannot  duplicate  a  reported 
symptom  during  the  fault  verification  or  diagnostic  process.  An  ideal  diagnostic  advisor  would 
recognize  a  CND  problem  after  repeated  reports  of  the  same  symptom  and  utilize  information 
on  all  rectifications  performed  previously  to  efficiently  recommend  an  isolation  and  rectification 
process.  Diagnostic  advisor  inaccuracies  may  occur  in  the  following  two  situations: 


1.  Inaccuracies  may  occur  when  the  maintenance  technician  does  not  perform  a  fault 
verification  step  and  proceeds  to  fault  isolate  based  solely  upon  the  reported  symptom.  The 
diagnostic  advisor  could  guide  the  technician  through  a  series  of  "passed"  tests  until  the 
implicated  set  contains  only  one  fault.  Upon  completion  of  the  rectification,  a  functional  check 
is  performed.  The  result  of  a  "passed"  functional  check  (equivalent  to  the  fault  verification  step 
that  could  not  be  performed  as  a  result  of  the  CND  condition)  leads  to  the  incorrect  conclusion 
that  the  rectification  performed  fixed  the  problem  that  caused  the  original  symptom.  This 
sequence  of  events  could  produce  high  Retest  OK  (RTOK)  rates  and  the  true  fault  may  not  be 
rectified. 

2.  Inaccuracies  also  may  occur  when  the  fault  verification  is  performed  prior  to 
diagnostics.  The  fault  verification  results  will  indicate  no  faults  are  found  in  the  system  and 
isolation  of  potential  faults  may  never  be  initiated. 


Intgimittcnt 

An  intermittent  fault  may  cause  a  test  to  fail  one  moment  and  pass  the  next,  with  no 
rectification  action  taken  by  the  maintenance  technician.  This  is  truly  a  fault  in  the  system 
under  test,  but  it  is  one  that  comes  and  goes  randomly.  If  a  test  that  spans  an  intermittent  fault 
should  pass,  this  can  eliminate  that  fault  from  consideration  and  result  in  an  incorrect  outcome. 
Further,  if  a  maintenance  technician  performed  a  recommended  action  on  a  good  component 
and  the  intermittent  fault  allowed  the  functional  check  to  pass,  the  diagnostic  system  would 
report  the  fault  as  being  fixed  by  the  rectification.  However,  the  true  fault  would  still  remain  in 
the  system  under  test. 
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Tests  That  Fix 


Performing  a  diagnostic  test  can  fix  or  alleviate  a  fault  in  some  cases.  For  example,  a 
bus  failure  might  be  the  result  of  high  resistance  between  contacts  due  to  corrosion  or  tarnish. 
A  diagnostic  test  which  requires  disconnecting  and  reconnecting  a  connector  may  burnish  the 
contacts,  remove  the  cause  of  the  high  resistance,  and  allow  a  test  to  pass.  Upon  completion  of 
the  next  rectification,  a  functional  check  will  be  required.  Given  this  situation,  the  functional 
check  will  pass  provided  the  rectification  does  not  insert  new  faults.  The  diagnostic  module 
would  incorrectly  report  alleviation  of  the  problem  due  to  the  unnecessary  rectification  and 
would  produce  a  RTOK  at  the  next  level  of  maintenance. 


Retention  of  Test  Results 

A  diagnostic  system  should  retain  results  of  all  tests  performed  and  reason  properly 
concerning  implicated  and  exculpated  faults  (potential  faults  that  have  been  eliminated  from 
consideration).  However,  a  fault  should  be  identified  for  rectification  once  it  is  isolated.  Even 
complicated  diagnostic  problems  require  many  recursive  processes. 


Access  Groups 

When  ranking  tests  or  rectifications,  a  diagnostic  advisor  should  consider  access 
groups  for  rectification  and  testing  time  efficiency.  An  access  group  is  a  collection  of  tests  or 
rectifications  revealed  by  a  single  maintenance  activity.  Once  access  to  an  area  is  gained, 
additional  activities  in  that  area  can  be  performed  at  very  little  cost.  Diagnostic  efficiency  may 
be  gained  when  actions  are  performed  within  access  groups  that  allow  the  user  to  gain 
additional  information  but  have  high  access  times. 


"ButNfll'LData  Entry 

When  entering  symptoms  for  fault  isolation,  the  absence  of  certain  symptoms  may 
exculpate  faults  intersecting  an  observed  symptom's  faults  (Table  1). 


Table  1.  "But  Not"  Data  Entry  Model 
Faults 

FI  F2  F3 _ E4 _ Ei 

Symptoms  S 1  1  1  1  0  0 

S2  0 _ CL-  -I _ 1  1 

For  example,  as  shown  in  Table  1,  symptom  SI  can  be  caused  by  faults  FI,  F2,  or  F3, 
and  S2  can  be  caused  by  F3,  F4,  or  F5.  If  the  diagnostic  module  is  designed  to  handle  "But 
Not"  data  entry  and  the  maintenance  technician  reports  that  SI  has  been  observed  but  not  S2, 
then  F3  can  be  exculpated.  If  SI,  entered  all  by  itself,  can  implicate  F3,  then  "But  Not" 
modeling  may  not  be  used. 
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Handling  of  Maintenance  Actions 


Maintenance  actions  are  rectifications  that  do  not  involve  removing  and  replacing 
(R&R)  components.  Rather,  they  require  adjustment  of  components  already  in  place. 
Diagnostic  advisors  should  process  the  results  (analyze  and  recommend  action)  of  maintenance 
actions  differently  from  those  of  R&R  actions.  For  example,  if  a  maintenance  action  fails  to 
rectify  certain  fault(s),  a  parent  action  then  needs  to  be  performed  before  a  functional  check  is 
performed.  Hence,  based  on  its  analysis  of  the  failed  maintenance  action,  the  diagnostic 
advisor  should  be  able  to  recognize  that  the  parent  action  must  be  taken. 


Criticality 

Aircraft  functions  which  are  required  for  the  next  planned  sortie  are  deemed  critical. 
This  is  extremely  important  in  terms  of  operational  requirements  requiring  tight  schedules  or 
quick  turnarounds  The  capability  to  distinguish  between  critical  and  noncritical  functions  is 
highly  desirable  in  diagnostic  advising  systems  since  the  presence  of  noncritical  faults  may  still 
enable  aircraft  utilization.  When  a  mode  of  operation  is  selected  which  implies  that  some  sets  of 
faults  are  noncritical,  a  diagnostic  module  may  attempt  to  test  or  rectify  critical  functions  early 
in  the  diagnostic  process.  A  well-designed  diagnostic  aid  should  attempt  to  either  implicate  or 
exonerate  the  entire  set  of  critical  faults  in  a  single  preferred  action.  Criticality  functions  may  be 
inefficient  in  at  least  three  areas: 

1 .  Immediate  repair  recommendations,  either  within  or  outside  the  critical  fault  set, 
can  lose  appeal  because  a  new  part  might  introduce  a  new  fault. 

2.  Tests  which  span  more  than  the  critical  fault  set  may  be  excluded  from 
consideration. 

3.  The  selection  of  critical  tests  or  actions  may  cause  diagnostic  inefficiencies  when 
compared  to  a  diagnostic  process  implemented  without  the  criticality  function 
activated  in  the  diagnostic  system. 


Data  Base  Issues 

There  are  several  opportunities  for  data  element  error  within  the  automatic  diagnostic 
tool:  incorrect  data  entry;  incorrect  manipulation  of  data  within  the  data  base;  or,  in  the  case  of 
newly  developed  c  mponents,  limited  availability  of  information  on  components,  tests,  or 
equipment.  In  view  of  this,  a  diagnostic  advisor  must  deal  effectively  and  efficiently  with 
incorrect  or  incomplete  supporting  data  bases. 

To  identify  possible  sources  of  error  that  may  hinder  the  diagnostic  activity  and  define 
the  extent  to  which  it  may  be  hindered,  one  must  (a)  identify  a  broad  array  of  data  elements  and 
maintenance  information,  (b)  determine  potential  sources  of  this  information,  and  (c)  identify 
uses  of  this  information  within  the  diagnostic  system.  Table  2  lists  data  elements  used  within 
the  IMIS  diagnostic  module  and  the  potential  sources  of  the  information.  The  evaluation 
during  this  effort  will  focus  on  necessary  uata  elements  directly  utilized  within  the  IMIS 
diagnostic  module. 
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Table  2.  Data  Elements 


Technical  Orders 

Fault 

Other 

Sources 

Data  Elements 

Isolation 

Manual 

FMECA& 
LSAR  Data 

Symptoms 

X 

X 

Faul^ 

X 

X 

Rectifications 

X 

X 

Tests 

Mission 

X 

X 

Aircraft  Flight  Manual 

Test  Equipment 

X 

X 

Job  Guide 

Repair  Level 

X 

RLA,  LORA 

MTBF 

Reference  Designator 

X 

X 

MODAS,  FMECA,  LSAR 

National  Stock  No. 

X 

Part  #/NSN  Reference 

List 

EfFectivity 

Fault  Weights 

Test  Times 

X 

Subject  Matter  Experts, 
Tech  Data 

X 

Subject  Matter  Experts, 
Tech  Data 

Rectification  Times 

X 

Subject  Matter  Experts, 
Tech  Data 

Configuration 

X 

Aircraft  Flight  Manual, 
Technical  Order,  Higher 

Headquarter  Operating 
Procedures,  Subject 

Matter  Experts 

Functional  Groups 

X 

Fault  Isolation  Manual 

Access  Groups 

X 

X 

Fault  Isolation  Manual 
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As  the  IMIS  process  matures,  information  will  be  readily  available  from  other  data 
bases  such  as  Air  Force  Technical  Order  Management  System  (AFTOMS),  Reliability  and 
Maintainability  Information  System  (REMIS),  and  Core  Automated  Maintenance  System 
(CAMS.)  Figure  2  illustrates  proposed  and  developing  data  bases  where  data  elements  for  use 
in  the  diagnostic  module  algorithms  will  be  available. 


The  identified  data  elements  may  be  categorized  into  two  groups  that  show  the  necessity 
of  the  information  gained  in  performing  diagnostic  analyses.  Table  3  depicts  two  groups  of 
IMIS  data  elements:  data  necessary  for  efficient  diagnostics  and  data  that  are  useful  but  not 
critical  to  the  IMIS  diagnostic  module. 


Table  3.  Data  Element  Utility 


Necessary _ Useful 


Symptoms 

Faults 

Rectifications 

Tests 

Mean  Time  Between 


Test  Equipment 
Repair  Level 
Reference  Designator 
National  Stock  Number 


Failures  (MTBF)a  Fault  Weights 

Test  Times  System  Configuration 

Rectification-Times _ Access  Groups _ 

a  Component  MTBF  data  are  used  to  create  fault 
weightings  if  fault  probability  information  is 
not  available. 


To  evaluate  the  diagnostic  system's  ability  to  complete  diagnostics  under  less  than 
optimal  data  availability  conditions,  several  scenarios  have  been  developed  that  define  potential 
problem  areas.  The  potential  problem  areas  include  inaccurate  data,  incomplete  data,  or 
inaccessible  maintenance  information.  Inaccurate,  incomplete,  and  inaccessible  maintenance 
information  data  discussions  will  be  confined  to  data  input,  retrieval,  and  analysis  problems 
within  the  diagnostic  advisor  and  supporting  data  bases. 


Inaccurate  Data 


The  scenarios  listed  below  will  test  potential  areas  in  which  the  diagnostic  system  may 
be  less  effective  due  to  inaccurate  data. 

Unseen  Fault.  An  unseen  fault  occurs  in  the  data  base  when  a  fault  is  implicated  by  a 
symptom  but  is  not  included  in  a  supporting  data  base.  What  happens  if  the  data  base  for  a 
given  fault  contains  a  weight  of  0%? 
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MAINTENANCE 

DIAGNOSTIC 

AIDING 

SYSTEM 


MDAS 


AIR  FORCE  TECHNICAL 
ORDER  MANAGEMENT 
SYSTEM 


ATOMS 

SYMPTOMS 

FAULTS 

RECTIFICATIONS 

TESTS 

FUNCTIONAL  GROUPS 
TEST  EQUIPMENT 
REFERENCE  DESIGNATOR 
ACCESS  GROUPS 


RELIABILITY  AND  MAINTAINABILITY 
INFORMATION  SYSTEM 


H 

BEM1S 

MEANTIME  BETWEEN 

i 

S 

FAILURES  (MTBF) 
WEIGHTS 

CONFIGURATION 

CORE  AUTOMATED 
MAINTENANCE  SYSTEM 

CAMS 

TEST  TIMES 

RECTIFICATION  TIMES 
CRITICALITY  OF  MISSIONS 
REPAIR  LEVEL 

Figure  2.  Data  Elements  Used  by  IMIS. 


Useless  Fault.  A  useless  fault  in  the  data  base  occurs  when  a  fault  is  not  truly 
implicated  by  a  symptom  but  is  inadvertently  included  in  the  supporting  data  base.  To  what 
extent  are  diagnostics  affected? 

Fault/Svmptom  Probability  Errors.  Probability  errors  may  occur  when  faults  are 
implicated  by  a  symptom  and  included  in  the  supporting  data  base  but  carry  incorrect 
probability  data.  To  what  extent  are  the  diagnostic  system's  efforts  compromised  by  incorrect 
probability  data? 


Incomplete  Data 

The  following  scenarios  describe  areas  in  which  a  diagnostic  system's  effectiveness 
may  be  diminished  because  of  incomplete  data. 


Unassigned  Fault  Weight/MTBF.  Given  neither  a  fault  weight  nor  an  MTBF,  how 
does  the  diagnostic  advisor  react? 


Unmodeled  Symptom.  An  unmodeled  symptom  is  produced  by  the  aircraft  system  but 
not  included  in  the  data  supporting  the  diagnostic  system.  The  supporting  data  base  must 
initially  depend  on  data  from  the  design  process  and  may  exclude  symptoms  that  will  appear 
after  the  equipment  is  fielded.  How  does  the  diagnostic  system  react  to  symptoms  not  included 
in  the  data  base? 

Unmodeled  Weights/MTBFs  for  Fault/Svmptom  Relationships.  Faults  may  be 
implicated  by  a  symptom,  but  weight  and/or  MTBF  data  may  be  missing.  In  the  event  of 
incomplete  data  lists,  either  weight  or  MTBF  but  not  both,  how  does  the  system  react? 


Accessibility  of.  Maintenance.  Information 

The  following  scenarios  describe  problems  that  could  create  difficulties  for  a  diagnostic 
advisor  because  of  an  incorrect  or  inaccurate  supporting  maintenance  information  data  base. 

Unmodeled  Tests  and  Rectifications.  An  observed  symptom  may  implicate  a  fault  for 
which  no  tests,  rectifications,  or  maintenance  actions  are  available  for  isolation  or  repair.  If  a 
fault  is  not  modeled  to  a  test,  rectification,  or  maintenance  action,  what  choices  of  action  does 
the  maintenance  technician  have  available? 

Unmodeled  or  Incorrect  Action  Times.  Test  and  rectification  times  are  key  elements  of 
analyses  that  determine  the  best  action.  When  not  available,  their  absence  may  pose  calculation 
problems.  Activity  values  in  the  IMIS  diagnostic  module  are  based  upon  information  gained 
per  unit  time.  Therefore,  a  zero  or  blank  value  for  activity  time,  if  passed  to  the  diagnostic 
algorithms  from  the  data  base,  may  result  in  system  failure.  How  does  the  diagnostic  system 
react  to  this  problem?  This  situation  could  be  a  result  of  the  lack  of  information  early  in  the  life 
cycle  of  a  weapon  system. 

Useless  Tests  and  Rectifications.  Inaccurate  fault-to-rectification  modeling  may  result  in 
a  rectification  action  that  does  not  actually  repair  the  fault  as  indicated  in  the  data  base.  When 
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faced  with  such  a  circumstance,  what  alternatives  are  available  to  the  diagnostic  system  and  the 
maintenance  technician? 


Inaccurate  fault-to-test  modeling  would  result  in  a  test  that  does  not  actually  span  the  set 
of  faults  that  are  indicated  in  the  data  base.  When  faced  with  such  a  circumstance,  what 
alternatives  are  available  to  the  diagnostic  system  and  the  maintenance  technician? 

Unseen  Test  Faults.  A  test  may  evaluate  faults  not  shown  in  the  data  base  as  being 
spanned  by  the  test  (faults  not  considered  in  the  table  relationship).  Therefore,  a  failed  test  may 
result  in  the  rectification  of  the  incorrect  component.  Can  the  diagnostic  system  handle  such  a 
situation  gracefully? 

Missing  Technical  Order  Information.  Incorrect  information  or  no  information  may 
appear  when  a  rectification  sequence  is  selected.  In  light  of  this,  what  options  are  available  to  a 
diagnostic  system  and  the  maintenance  technician? 


User  Interface  Issues 

Six  user  interface  scenarios  have  been  defined  to  test  a  diagnostic  system's  ability  to 
deal  with  potential  real  world  problems:  (a)  incorrect  user  inputs,  (b)  reentry  of  fault/symptom 
information  after  a  functional  check,  (c)  handling  of  data  bus  outputs,  (d) 
initialization/reinitialization  of  the  diagnostic  session,  (e)  return  of  more  than  one  outcome  from 
a  Multiple  Outcome  Test  (MOT),  and  (f)  rectification  and  test  sequencing. 


Incorrect  UssUnput 

Correct  user  input  is  of  major  importance  to  the  efficiency  and  effectiveness  of 
diagnostic  activity.  The  IMIS  diagnostic  module’s  user  inputs  are  listed  in  Table  4,  and  relative 
impact  of  their  effect  on  efficient  diagnostics  is  shown.  "Crucial"  user  inputs  represent  choices 
that,  when  performed  incorrectly,  may  corrupt  maintenance  aiding  ability  with  no  hope  of 
recovery.  "Important"  inputs  represent  user  interface  selections  that,  if  made  incorrectly,  will 
lead  to  inefficient  diagnostics.  "Not  important"  choices  do  not  initiate  diagnostic  activity  within 
the  diagnostic  system. 

Criticality.  Diagnostic  activity  might  be  inadvertently  or  incorrectly  initiated  with 
criticality  invoked  but  mission  criticality  is  of  little  importance.  Can  adverse  effects  be  exhibited 
when  the  actual  fault  is  not  critical? 

Symptom.  Incorrect  user  input  for  symptoms  could  take  two  forms,  both  of  which 
might  be  fatal  if  the  incorrect  entry  is  not  discovered  by  the  maintenance  technician.  In  the  two 
scenarios  below,  can  changes  to  the  status  create  difficulties  in  diagnostic  analyses?  Are  there 
any  restrictions  as  to  when  these  changes  can  be  made? 

1.  Incorrectly  marked  symptom.  This  event  occurs  when  a  symptom  is  designated 
but  is  not  an  exhibited  symptom. 

2.  Unmarked  symptom.  This  event  results  when  a  symptom  is  exhibited  by  the 
system  under  test  but  is  not  designated  for  diagnostic  analyses. 


Table  4.  User  Input  Criticality 


User  Inputs 

Not 

Important 

Important 

Crucial 

Criticality 

X 

Symptoms 

Initialization  Input 

X 

Re-initialization  (remove,  add) 

X 

Test  Choice 

X 

Rectification  Choice 

X 

Test  Results 

X 

Funct.  Chk.  Results 

X 

Exiting  MDAS 

X 

Display  Tests 

X 

Display  Rectifications 

X 

Display  Actions 

X 

ETIC 

X 

Look  Ahead 

X 

Test  Results.  There  are  two  scenarios  which  provide  insights  to  diagnostic  advisor  reactions 
when  faced  with  incorrect  test  result  entries: 

1 .  Single  incorrect  entry.  This  situation  results  when  a  single  incorrect  test  outcome 
entry  is  made.  Does  the  diagnostic  advisor  provide  the  capability  to  back  up  in  die 
event  of  a  single  incorrect  entry?  What  results  are  observed  when  the  incorrect  test 
result  is  not  reentered,  and  can  diagnostics  be  completed  with  the  rectification  of  the 
true  fault? 

2.  Multiple  incorrect  entries.  Occasionally,  a  new  maintenance  technician  may  forget 
diagnostic  intricacies  or  lack  knowledge  of  test  outcomes.  In  this  situation,  a 
maintenance  technician  may  enter  incorrect  test  outcomes  with  consistency.  Can  the 
diagnostic  advisor  recover  or  identify  such  a  situation? 


Functional  Check  Results.  Two  scenarios  that  provide  insights  to  diagnostic  system 
reactions  when  faced  with  incorrect  entry  of  functional  check  results  are  shown  below.  These 
are  similar  to  the  test  result  scenarios  shown  above,  but  the  effects  may  be  very  different. 

1 .  Single  incorrect  entry.  This  situation  occurs  when  a  single  incorrect  functional 
check  outcome  entry  is  made.  Can  the  diagnostic  advisor  be  given  the  capability  to 
back  up  in  the  event  of  a  single  incorrect  entry?  What  options  are  available  after  an 
incorrect  entry,  and  can  diagnostics  be  completed  successfully? 

2.  Multiple  incorrect  entries.  At  times,  a  new  maintenance  technician  may  enter 
incorrect  functional  check  outcomes  consistently.  Diagnostic  advisors  are  dependent 
on  correct  user  inputs  for  functional  checks  and  may  not  be  able  to  recover  from 
this  situation. 


The  results  of  a  functional  check  can  be  entered  into  the  diagnostic  system  either 
automatically  or  manually.  Automatic  feedback,  via  an  aircraft  data  bus,  initializes  built-in  tests 
(BITs)  and  returns  symptoms  which  are  directly  input  to  diagnostic  activity.  Could  these 
methods  allow  symptoms  to  be  inadvertently  alleviated,  although  not  tested,  or  could  newly- 
observed  symptoms  be  overlooked? 


A  demonstration  of  the  IMIS  portable  computer  coupled  with  the  presentation  and 
diagnostic  modules  was  accomplished  on  an  F-16  aircraft  during  1989.  The  MIL-STD-1553 
data  bus  was  important  in  downloading  system  and  fault  information  for  use  by  the  diagnostic 
module.  The  Fire  Control  Computer  (FCC),  via  the  MIL-STD-1553  data  bus,  initiated  system 
and  fault  checks,  and  received  and  stored  information  from  BITs.  This  information  was  used 
for  both  fault  verification  and  initializing  diagnostics.  Figure  3  illustrates  the  MIL-STD-1553 
data  bus  with  some  system  connections  and  shows  its  usefulness.  The  block  diagram  in  Figure 
4  displays  the  flow  of  diagnostics  and  fault  information. 

Since  the  MIL-STD-1553  data  bus  is  so  important  in  fault  verifying  and  initializing  the 
diagnostics  process,  what  outputs  can  be  expected  of  the  portable  maintenance  aid,  and  what 
are  the  effects  of  incorrect  outputs  from  the  MIL-STD- 1 553  data  bus? 
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An  important  first  step  for  any  diagnostic  system  should  be  to  verify  the  existence  of  a 
fault  in  the  system  under  test.  Current  human  interface  design  for  the  IMIS  diagnostic  module 
initializes  the  sequence  based  upon  the  pilot-repented  discrepancies  contained  in  the  Air  Force 
Technical  Order  (AFTO)  Form  349  (shown  in  Figure  4  as  "debrief').  However,  the  IMIS 
diagnostic  module  does  not  call  fen-  a  fault  verification  procedure  before  starting  the  diagnostic 
routine.  Based  on  the  flow  of  information  from  Figure  4,  what  are  the  potential  pitfalls  of 
continuing  in  this  fashion,  and  what  alternatives  are  available  within  the  diagnostic  module  that 
can  provide  an  efficient  and  effective  initialization/reinitialization  without  fault  verification? 


FCC  SMS  TISL 

Fire  (Control  Computer  Stores  Management  System  Target  Illuminating  System 

Laser 
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Figure  4.  Flow  of  Diagnostics  Information. 


As  opposed  to  binary  tests  which  have  only  two  outcomes  (pass/fail),  MOTs  are  tests 
that  have  three  or  more  possible  outcomes. 


There  are  three  ways  to  author  and  perform  MOTs: 

1 .  Exit  at  first  failure.  The  maintenance  technician  reports  the  first  failed  test  outcome 
only,  performs  the  associated  activities,  and  starts  the  test  over  again.  This 
process  continues  until  the  MOT  is  completed  with  no  failures. 

2.  Complete  the  MOT  and  report  all  failures  encountered. 

3.  Complete  the  MOT  and  only  report  one  failure. 

A  diagnostic  tool  can  be  designed  to  work  with  any  of  the  three  methods  or  all  three.  If 
designed  for  one  of  the  three,  what  would  be  the  effect  if  a  test  authored  for  a  different  method 
were  encountered? 


Technical  Order  Sequencing 

While  the  presence  or  absence  of  a  correct  technical  order  procedure  for  performing  a 
required  test  or  maintenance  action  is  a  straightforward  data  base  issue,  there  are  two  rather 
more  complex  technical  order  sequencing  functions  which  must  be  treated  properly  by  the  user 
interface  if  diagnostics  are  to  be  completed  successfully: 

Facilitate  Other  Maintenance  (FOM).  In  the  event  that  Component  A  must  be  removed 
to  gain  access  to  Component  B,  does  the  technical  order  presentation  system  provide  the  proper 
procedures  in  the  proper  sequence?  Do  the  rectification  and  testing  sequences  reflect 
reinstallation  of  all  components  (A  and  B)  necessary  for  functional  check  completion? 

Cannibalization  If  components  are  not  currently  available  from  supply,  cannibalization 
of  another  aircraft  may  be  necessary  to  complete  fault  rectification  or  isolation.  Does  the 
technical  order  presentation  system  provide  the  capability  to  sequence  cannibalization 
instructions  properly  between  the  donor  and  receiver  aircraft? 


m.  SYSTEM  DESCRIPTION 

The  scenarios  developed  for  testing  the  IMIS  diagnostic  module  are  general  and  may  be 
applicable  to  many  diagnostic  systems.  However,  in  many  cases,  the  test  results  are  specific  to 
the  integrated  system  developed  for  the  IMIS  field  demonstration.  This  system  integrates  the 
diagnostic  module  into  a  technical  data  APS.  Consequently,  a  brief  description  of  the  structure 
and  operation  of  the  integrated  system  is  warranted  before  presenting  test  results. 

The  basic  structure  of  the  tested  system  is  shown  in  Figure  5.  The  authoring  system  is 
used  to  create  the  technical  data  processed  through  the  diagnostic  module  and  the  presentation 
system  and  ultimately  displayed  to  the  maintenance  technician  at  the  user  interface.  The 
diagnostic  module  obtains  inputs  from  the  user  interface  and  data  from  the  integrated  data  base 


to  develop  recommendations  for  fault  isolation/rectification  sequencing  and  provides 
recommendations  for  display  to  the  user.  The  presentation  system  obtains  recommendations 
from  the  diagnostic  module  and  technical  order  data  from  the  integrated  data  base  as  well  as 
aircraft  status  data  from  the  MIL-STD-1553  data  bus.  The  presentation  system  provides 
sequenced  rules,  procedures,  graphics,  and  other  information  to  the  maintenance  technician  ami 
processes  feedback  from  both  the  aircraft  and  the  technician  to  define  the  next  display 
requirement.  In  addition,  the  presentation  system  employs  programmable  function  keys  to 
assist  the  technician  in  navigating  through  the  required  technical  orders  and  other  procedures. 


Figure  5.  Integrated  IMIS  Demonstration  System. 

Key  data  requirements  for  the  diagnostic  module  are  contained  in  both  the  technical 
order  data  and  the  diagnostic  data.  The  technical  order  data  provides  test  and  rectification 
procedures  and  times  to  accomplish  these  procedures.  The  diagnostic  data  base  contains  a 
complete  list  of  faults  associated  with  the  system  under  test  and  fault  mapping  tables  which 
map  faults  to  symptoms,  tests,  and  rectifications.  Data  entry  for  faults  includes  both  the  fault 
MTBF  (mandatory)  and  a  fault  weight  related  to  a  particular  symptom  (optional). 

The  structure  of  the  diagnostic  module  is  shown  in  Figure  6.  Fault,  symptom,  test,  and 
rectification  data  for  the  system  under  test  are  loaded  along  with  supplemental  information 
regarding  criticality  (on  or  off  by  function),  part  availability  from  supply,  non-standard  aircraft 
configuration  identification,  and  the  symptom(s)  which  initiated  the  diagnostic  effort  Based 
upon  this  information,  the  module  calculates  three  lists: 

1 .  A  "Best  Test"  list  providing  a  ranked  list  of  available  tests  useful  in  isolating  the 
reported  fault. 

2 .  A  "Best  Action"  list  providing  a  ranked  list  of  maintenance  actions  which  may  be 
useful  in  resolving  the  reported  symptom. 

3 .  A  "Best  Options"  list  providing  a  "top  five"  integrated  ranked  list  of  both  tests 
and  repairs.  The  top  ranked  test  or  action  from  this  list  is  presented  to  the  user  as 
the  diagnostic  module's  recommended  activity. 


15 


Figured.  Logic  Flow. 
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The  user  may  select  any  activity  from  any  of  the  lists  and  the  presentation  system  will 
provide  the  required  technical  order  instructions  for  performing  the  activity.  Upon  completion 
of  the  activity,  results  are  returned  to  the  diagnostic  module  and  the  processing  continues  as 
necessary  until  the  system  is  returned  to  operational  status. 


IV.  IMIS  DIAGNOSTIC  MODULE  TEST  RESULTS 

Exhaustive  test  procedures  were  developed  to  test  the  IMIS  diagnostic  module  against 
each  of  the  scenarios  defined  in  Section  II.  This  section  of  the  paper  provides  results  of  the 
tests  performed  upon  the  IMIS  diagnostic  module.  Detailed  test  procedures  and  results  from 
each  test  are  provided  in  the  Appendix  to  this  paper.  The  tests  were  accomplished  on  a  Sun  II 
workstation  using  Version  3.3  of  the  IMIS  diagnostic  and  technical  order  APS  modules. 
Parameter  values  were  varied,  and  memory  values  were  observed  using  the  dbxTMl  facility 
provided  in  the  Sun  version  of  the  UNIX  operating  system.  The  results  of  the  testing  and 
analysis  of  test  results  provide  the  basis  for  the  recommendations  contained  in  Section  V. 


Fault  Processing-Issues 

These  tests  show  that  certain  fault  processing  complications  can  cause  problems  for  the 
diagnostic  module,  ranging  from  inefficiencies  to  inability  to  help  in  fault  rectification. 


Can  Not  Duplicate  (CND) 

The  diagnostic  module  did  not  easily  handle  CND  problems.  There  were  two  CND 
problems  examined  during  testing.  The  first  CND  problem  stems  from  a  transient  condition 
which  has  cleared  itself  and  no  longer  exists  in  the  system.  If  fault  verification  is  performed 
prior  to  starting  the  diagnostic  module,  there  will  be  no  degradation  of  diagnostics.  On  the 
other  hand,  if  fault  verification  is  not  performed  prior  to  starting  the  diagnostic  module, 
diagnostics  will  lead  down  a  series  of  "passed"  tests  until  only  one  fault  remains,  and  then  a 
rectification  will  be  performed  "successfully."  This  will  lead  to  an  RTOK  message  at  the 
intermediate  level  shop. 

The  second  problem  examined  stems  from  a  situation  where  there  really  is  a  fault  in  the 
system,  but  no  test  can  "see"  it.  Any  test  run  on  the  system  will  pass,  so  the  result  will  be  the 
same  as  the  unverified  CND  explained  above.  The  diagnostic  module  will  be  of  dq  assistance 
in  this  situation,  and  the  fault  could  only  be  repaired  if,  by  fortuitous  accident,  it  occurred  in  the 
component  replaced  after  the  series  of  tests. 


IntermitteniFault 

The  IMIS  diagnostic  module  was  not  effective  in  handling  this  fault  processing 
problem.  If  the  intermittent  fault  happens  to  be  present  during  the  diagnostic  process,  then  it 
could  be  handled  the  same  as  any  verified  fault.  If  it  disappears,  then  it  looks  very  much  the 
same  as  the  CND  problem. 


^dbxTM .  sun  Work  Station  (R)  Sun  Microsystems  (Revision  A  of  17  February  1986) 
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Tests  That  Fix 


This  problem  creates  a  situation  that  behaves  much  like  a  CND  problem  but  occurs  after 
diagnostics  have  started  with  a  verified  fault.  It  will  result  in  incorrect  fault  identification  and 
an  RTOK. 


Tests  That  Fix 

This  problem  creates  a  situation  that  behaves  much  like  a  CND  problem  but  occurs  after 
diagnostics  have  started  with  a  verified  fault.  It  will  result  in  incorrect  fault  identification  and 
an  RTOK. 

Loss  of  Test  Information 

Fault  handling  procedures  in  the  diagnostic  module  resulted  in  the  loss  of  test 
information.  In  the  test,  the  human  operator  could  see  that  two  simultaneous  faults  had  been 
isolated  to  two  specific  components;  yet,  the  system  returned  a  test  recommendation.  Fault 
isolation  was  ultimately  successful,  but  the  most  efficient  strategy  was  not  recommended  (see 
Test  #5  in  the  Appendix). 

Access  Groups 

The  access  group  algorithms  are  not  incorporated  in  the  current  diagnostic  module  and 
could  not  be  tested. 


"But  Not"  Data  Entry 

The  IMIS  diagnostic  module  is  not  designed  to  take  advantage  of  the  additional 
information  provided  by  the  concept  of  "But  Not"  data  entry.  Consequently,  faults  which 
might  be  removed  from  consideration  during  initialization  must  be  treated  as  "possibles"  and 
considered  during  fault  isolation.  Although  the  "But  Not"  facility  is  not  built  into  the  diagnostic 
module,  the  result  is  inefficient  diagnostics  (extra  steps  required)  rather  than  a  failure  of 
diagnostics. 

Handling-flE  Maintenance  Actions 

The  diagnostic  module  did  not  distinguish  between  a  maintenance  action-which  does 
not  involve  a  removal  or  replacement  --  and  other  repair  activities.  Consequently,  as  the  faults 
were  processed,  the  system  entered  a  closed  loop  wherein  the  same  sequence  of  tests  and 
actions  was  called  repeatedly.  Diagnostics  were  unsuccessful. 

Criticality 

Selecting  criticality,  as  done  in  this  test,  resulted  in  inefficient  diagnostics  and  delayed 
the  total  time  to  either  exculpation  of  the  critical  faults,  rectification  of  the  critical  faults,  or 
rectification  of  the  existing  noncritical  fault.  The  criticality  function  implemented  in  this  version 
of  the  IMIS  diagnostic  module  was  inefficient. 


Pata  Pass  Issues 


The  elements  of  the  data  base  and  the  acquisition  and  use  of  the  data  base  elements  are 
crucial  to  proper  functioning  of  the  IMIS  diagnostic  module.  This  section  describes  the  results 
of  testing  that  examined  the  effect  of  a  corrupted  data  base  on  the  proper  operation  of  the 
diagnostic  module.  The  error  sources  examined  included  h^man  error  in  entering  data,  machine 
(software)  error  in  transferring  data  from  the  data  entry  fonh  to  the  digital  data  base,  and  failure 
to  enter  required  data  in  either  the  diagnostics  data  base  or  the  technical  order  data  base. 
Results  from  testing  the  data  base  scenarios  are  recounted  below. 

Inaccurate  Data 


Unseen  Fault.  A  weight  value  of  zero  was  assigned  to  a  fault  and  a  symptom  which 
implicated  that  fault  was  selected  as  the  observed  symptom.  Such  an  incorrect  entry  might 
result  from  lack  of  knowledge  about  a  fault’s  weight  during  data  entry  or  the  assignment  of 
zero  as  a  default  value. 


When  the  value  of  zero  was  received  by  the  diagnostic  module  for  a  fault/symptom 
weight,  the  fault  was  ignored  in  all  diagnostic  displays.  Tests  and  rectifications  for  the  affected 
fault  were  not  displayed  and  could  not  be  performed. 

Useless  Fault.  A  useless  fault  was  defined  in  the  scenarios  section  as  one  which  was 
shown  in  the  data  base  as  implicated  by  a  symptom,  when,  in  fact,  the  fault  had  nothing 
whatever  to  do  with  that  symptom.  The  test  showed  that  including  such  a  fault  in  the  data  base 
can  reduce  diagnostic  efficiencies  but  does  not  impede  the  success  of  the  diagnostic  effort.  If  a 
very  low  MTBF  accompanies  the  incorrect  fault  entry  in  the  data  base,  there  is  a  very  real 
possibility  that  an  RTOK  could  also  result. 

Fault/S vmptom  Probability  Errors 

1.  Incorrect  weights/  weights  do  not  sum  to  100.  The  data  base  was  loaded  with  the 
weights  for  the  faults  in  a  symptom  summing  to  73%,  and  that  symptom  was  selected.  Even 
though  weights  did  not  sum  to  100,  diagnostics  continued  undisturbed.  Correct  rectifications 
were  performed  based  on  the  probabilities  assigned  in  the  table.  In  conclusion,  if  weights  are 
in  error,  the  diagnostic  module  does  not  have  the  capability  to  detect  deviations,  and 
diagnostics  cnunue  normally  with  respect  to  the  weights  assigned. 

2.  Incorrect  MTBFs  (MTBF  =  infinite’).  In  the  data  base,  the  fault  MTBFs  were 
assigned  as  follows:  F0=10,  Fl=9999=largest  number  that  can  be  entered  into  the  data  base 
for  an  MTBF  value,  and  F2=10.  When  this  situation  was  tested,  R1  (the  corresponding 
rectification  for  FI)  displayed  zero  in  the  options  list,  but  the  actual  memory  value  was 
.000499.  Results  of  this  test  showed  that  no  information  is  lost  as  a  result  of  rounding  off 
numbers  in  the  options  list,  and  all  tests  and  rectifications  were  displayed  in  the  appropriate 
ranking  with  resnect  to  their  MTBF  values.  Diagnostics  were  completed  successfully. 

3.  Incorrect  MTBFs  (MTBF  =  zero’!.  The  lowest  MTBF  value  that  could  be  assigned 
in  the  data  base  for  an  MTBF  was  zero.  A  fault,  FI,  was  assigned  a  zero  MTBF  value,  and 
diagnostics  were  performed.  The  result  of  a  zero  MTBF  was  devastating  to  the  diagnostic 
module.  Processing  errors  occurred,  and  all  other  faults  were  removed  from  consideration. 
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Only  tests  and  rectifications  for  FI  were  considered.  Additionally,  rectifications  associated 
with  FI  had  option-list  probabilities  of  214%.  The  memory  location  for  FI  contained  "NaN" 
which  is  an  out-of-  bounds  value  that  signifies  "not  a  number." 


Incomplete  Data 


Unassigned  Fault  (Weight/MTBFl  Value 

1.  Weights.  In  the  data  base,  no  weight  value  was  assigned  for  FI  (the  data  base  form 
permits  a  blank  entry  for  input  of  weight  values).  The  operating  system  returned  a  value  of 
65535,  and  diagnostics  continued  by  accessing  the  MTBF  as  a  default  value.  Unassigned 
weight  values  pose  no  problems  with  diagnostics  if  MTBF  values  are  available.  The  only 
inefficiency  that  exists  in  such  a  scenario  corresponds  to  the  assignment  of  fault  probability 
values  calculated  from  MTBFs. 

2.  MTBFs.  The  data  base  form  requires  a  positive  integer  or  zero  value  for  an  MTBF 
entry;  therefore,  undefined  entries  in  the  data  tables  cannot  exist  and  do  not  pose  diagnostic 
difficulties. 


Unmodeled  Symptom.  A  given  symptom  for  the  system  under  test  might  occur  but  not 
be  included  in  the  diagnostic  data  base.  Such  an  event  could  be  the  result  of  oversight  when 
developing  the  data  base  or  the  result  of  lack  of  knowledge  early  in  a  weapon  system  life  cycle. 
Whether  the  symptom  was  observed  by  the  maintenance  technician  (manual  symptom  entry 
required)  or  through  download  from  the  MIL-STD-1553  bus  interface  (automatic  symptom 
entry),  the  result  was  the  same.  The  observed  symptom  has  to  be  matched  to  an  existing 
symptom  list  in  the  data  base.  When  the  observed  symptom  could  not  be  found  in  the 
symptom  list,  there  was  no  way  to  enter  the  observed  symptom  or  obtain  fault  information 
associated  with  that  symptom.  If  a  fault  were  implicated  solely  by  the  unmodeled  symptom, 
the  IMIS  diagnostic  module  would  not  be  useful  in  fault  isolation  and  repair. 

Unmodeled  Weights/MTBFs  for  Fault/Svmptom  Relationship.  The  data  base  may 
contain  errors  wherein  a  fault  implicated  by  a  symptom  may  have  weight  and/or  MTBF  values 
missing.  The  first  situation  tested  considered  the  absence  of  both  MTBF  and  weight  values  for 
the  same  fault.  When  diagnostics  were  initialized,  processing  errors  occurred  and  diagnostics 
were  not  completed. 

The  second  entries  made  in  the  data  base  involved  absence  of  fault  weight  and  MTBF 
values  in  different  faults.  Since  one  weight  was  blank,  the  diagnostic  module  reverted  to 
MTBFs  for  calculation  of  that  weight.  The  weight  value  for  the  other  fault  was  utilized,  and  its 
MTBF  was  not  accessed.  Diagnostics  continued,  completing  all  rectifications. 


Accessibility  of  Maintenance  Information 

Unmodeled  Tests  and  Rectifications.  An  observed  symptom  implicates  a  fault  for 
which  no  tests  or  rectifications  are  available.  Test  and  rectification  mapping  were  removed 
from  a  fault  to  simulate  a  fault  that  has  not  been  modeled  for  test  or  rectification  relationships. 
The  fault  was  included  in  the  fault/symptom  relationship  table,  but  as  it  was  not  modeled  to  a 
test  or  rectification,  diagnostics  continued  rectifying  or  testing  all  other  faults  until  the  corrupted 
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fault  was  the  only  one  left  in  the  plausible  set.  Diagnostics  ended  with  processing  errors 
because  no  rectification  or  test  link  to  technical  data  was  available  for  the  final  remaining  fault 

Unmodeled  or  Incorrect  Action  Times.  Unmodeled  and  incorrect  action  times  were 
tested  by  assigning  obviously  incorrect  times  to  various  test  and  rectification  activities  in  the 
data  base.  These  included  blank  entries,  zero  entries,  and  very  large  (99,999)  entries.  Blank 
entries  in  the  data  base  produced  the  same  results  as  zero  entries.  The  test  and  action  values  for 
the  items  affected  were  corrupted,  but  the  tests  and  actions  were  available  in  the  diagnostic 
system,  and  faults  could  be  successfully  isolated  and  repaired.  These  entries  resulted  in 
successful  but  inefficient  diagnostics. 

Entry  of  very  large  action  times  presented  no  problems  to  the  diagnostic  system. 
Obviously,  the  tests  and  rectifications  inevitably  ended  up  on  the  bottom  of  every  ranked  list  of 
available  activities  because  of  the  excessive  time  required  to  perform  the  activity.  Assuming  the 
very  large  value  for  time  was  in  error,  the  only  impact  was  inefficient  diagnostics. 

Useless  Tests  and  Rectifications.  This  test  was  performed  by  remapping  faults  in  the 
diagnostic  data  base.  A  fault  was  mapped  to  an  incorrect  rectification.  Each  time  that  fault  was 
isolated,  the  incorrect  rectification  was  called,  and  the  subsequent  functional  check  failed 
because  the  fault  was  still  present.  The  diagnostic  system  entered  an  infinite  loop  of  ineffective 
repairs  and  tests.  When  the  same  fault  was  mapped  to  an  incorrect  test,  the  test  passed,  thus 
exculpating  the  fault.  When  only  one  fault  (not  the  right  one)  remained  in  the  plausible  set,  the 
recommended  rectification  did  not  fix  the  fault,  the  functional  check  failed,  and  an  infinite  loop 
was  entered.  The  diagnostic  module  was  not  successful  when  facing  this  sort  of  data  base 
error. 


Unseen  Test  Faults.  This  test  was  performed  by  implicating  a  fault  not  contained  in  the 
data  base.  The  fault  was  spanned  by  a  test  which  also  spanned  other  faults.  When  the  test 
failed  due  to  the  unmodeled  fault,  an  unsuccessful  rectification  was  performed,  the  functional 
check  failed,  and  the  diagnostic  system  entered  an  infinite  loop.  The  IMIS  diagnostic  module 
was  ineffective  in  handling  this  type  of  data  base  problem. 

Missing  Technical  Order  (TOl  Information.  Two  types  of  data  base  problems  were 
associated  with  missing  technical  order  information.  A  call  from  the  diagnostic  module  to  the 
technical  order  data  base  was  linked  to  the  wrong  technical  instruction.  In  this  case, 
diagnostics  could  not  be  performed  because  there  was  no  way  to  obtain  correct  instructions  for 
performing  tests  or  rectifications.  The  other  problem  was  created  by  removing  the  link  from 
the  procedure.  In  this  test,  a  computer  core  dump  occurred  because  there  were  no  data  at  the 
address  specified  by  the  diagnostic  module.  In  any  case,  the  diagnostic  module  could  not 
successfully  overcome  errors  created  in  the  technical  order  data  base. 


Human  Interface  Issues 


Incorrect  User  Input 

In  order  to  gain  full  understanding  of  the  effects  of  user  input  errors,  erroneous  data 
were  entered  into  the  diagnostic  module.  Several  critical  user  interface  errors  were  identified 
when  the  following  tests  were  accomplished. 
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Criticality.  Upon  system  initialization,  criticality  was  invoked.  The  diagnostic  module 
recommended  a  critical  test.  When  the  test  passed,  all  critical  faults  were  exculpated  and 
diagnostics  ended  with  "Failed  Faults  Found."  At  this  point,  accrued  testing  and  rectification 
information  could  not  be  utilized  for  further  isolation  and  repair  of  noncritical  faults.  Criticality 
can  be  deselected  in  order  to  return  to  normal  diagnostics  any  time  before  all  critical  faults  are 
exculpated. 

Symptom.  To  investigate  possible  symptom  user  interface  errors  that  can  occur,  a  fault 
was  selected  as  the  true  fault,  and  an  incorrect  symptom  was  entered  as  the  observed  symptom. 
Testing  showed  that  symptoms  can  be  changed  at  any  point  in  the  diagnostic  process,  but  when 
a  symptom  is  eliminated,  all  its  faults  are  exculpated.  When  incorrect  symptom  entries  were 
not  found  and  corrected  or  when  symptoms  were  removed  from  consideration  too  soon,  all 
faults  associated  with  these  symptoms  were  exculpated.  They  could  not  be  recovered  with  the 
backup  function. 

Incorrect  Test  Results  Entry.  Incorrect  entry  of  test  results  by  the  technician  can  be  the 
result  of  either  error  or  failure  to  understand  the  questions  posed  by  the  user  interface.  In  any 
case,  an  incorrect  test  result  entry  can  have  very  different  effects  upon  the  diagnostic  module, 
depending  upon  the  type  of  test  being  run.  When  a  single  incorrect  "fail"  result  was  entered  for 
a  test  (not  a  functional  check),  the  diagnostic  module  recovered  quickly,  and  the  only 
degradation  was  in  inefficient  diagnostics.  When  a  single  incorrect  "pass"  result  from  a  test 
was  entered,  the  diagnostic  module  failed,  and  successful  fault  isolation  was  not  possible. 
Either  of  the  two  situations  could  be  corrected  by  striking  the  "Back  Up"  key  sufficient  times  to 
back  up  beyond  the  point  where  the  input  error  occurred. 

When  the  test  for  which  an  incorrect  entry  was  made  was  a  functional  check,  slightly 
different  results  were  noted.  An  incorrect  or  inadvertent  "pass"  entry  resulted  in  display  of 
"Failed  Faults  Found"  and  the  system  exited  diagnostics.  There  was  no  opportunity  to  correct 
the  entry.  The  only  choice  available  was  to  start  over.  An  incorrect  or  inadvertent  "fail"  entry 
had  little  effect.  The  diagnostic  module  eventually  recovered.  The  only  existing  problem  was 
inefficient  diagnostics.  However,  the  incorrect  entry  could  be  corrected  using  the  backup 
capability. 


Reentry  of  Fault/Svmptom  Information  After  a  Functional  Check 

Although  the  capability  for  the  IMIS  portable  computer  to  interface  with  the  1553  bus 
has  yet  to  be  implemented  for  diagnostic  initialization  and  fault  verification,  instances  can  be 
theorized  to  depict  problems  that  may  occur  when  the  results  of  a  functional  check  are 
automatically  entered  into  the  diagnostic  module. 

When  automatic  reentry  of  symptoms  occurs,  the  reinitialization  of  diagnostics  may 
exclude  symptoms  previously  observed  that  were  not  included  in  the  functional  check.  The 
automatic  reentry  could  also  exclude  manual  symptom  entries  which,  when  not  discovered  by 
the  maintenance  technician,  would  go  unnoticed.  In  either  case,  rectification  of  all  faults  may 
not  be  completed. 

Futhermore,  a  corrupted  1553  bus  output  could  be  an  unrecognizable  symptom  code  or 
could  match  an  existing  symptom  code.  If  the  corrupted  output  resulted  in  an  unrecognizable 
symptom  code,  the  diagnostic  module  could  not  find  the  implicated  fault  set  If  the  corrupted 
1553  bus  output  matched  an  existing  symptom  code,  the  diagnostic  module  would  initiate  fault 
isolation  on  an  incorrect  set  of  faults. 
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Return  of  More  Than  One  Outcome  From  a  MOT 


The  diagnostic  module  allows  only  one  test  outcome  entry  from  a  MOT.  When  an 
outcome  is  entered,  all  faults  spanned  by  the  test  but  not  implicated  by  that  outcome  are 
exculpated.  The  test  performed  on  this  function  was  concerned  with  the  effects  on  the 
diagnostic  module  if  a  MOT  authored  for  "complete  test  and  enter  all  outcomes"  is  encountered. 
When  the  test  was  run,  the  diagnostic  system  started  recalculation  of  the  plausible  set 
immediately  after  entry  of  the  first  outcome  Faults  from  subsequent  planned  outcome  entries 
were  exculpated  and  could  not  be  recovered  for  fault  isolation.  Since  the  functional  check  could 
not  pass  because  faults  were  incorrectly  exculpated,  the  system  could  not  successfully  complete 
fault  isolation  and  repair. 


Technical  Order  Sequencing 


SEI  attempted  technical  order  sequencing  for  two  very  difficult  problems:  Facilitate 
Other  Maintenance  (FOM)  and  cannibalization.  FOM  and  cannibalization  are  encountered  fairly 
often  during  diagnostic  sessions.  The  detailed  results  of  these  attempts  are  shown  in  the 
Appendix.  The  presentation  system,  which  assists  the  technicians  in  obtaining  correct  technical 
order  instructions,  was  unsuccessful  in  solving  these  problems.  In  both  problems,  the  attempt 
to  obtain  needed  instructions  eventually  produced  an  incorrect  "Failed  Faults  Found" 
termination  of  the  diagnostic  problem  before  the  actual  diagnostic  problem  was  more  complete. 


Errors  or  Inefficiencies  in  Normal  Diagnostics 


Nonfunctional  Kev  Entries.  When  performing  normal  diagnostic  sequences,  several 
key  entries  are  required  to  provide  the  diagnostic  module  with  information  about  the  selections 
and  the  outcomes  of  the  actions.  The  "active"  keys  are  depicted  on  the  user  interface.  When  a 
"non-active"  key  was  depressed,  errors  occurred  and  diagnostics  "bombed."  Nonfunctional 
key  entries  are  devastating  to  diagnostics. 

Recalculation  of  List  Options.  Best  tests,  rectifications,  and  actions  are  three  options 
used  extensively  to  view  and  select  ranked  actions.  Each  list  requires  extensive  calculations  be 
performed  when  diagnostics  are  initialized.  In  the  process  of  performing  validation  and 
verification  on  large  data  bases,  ranked  option  lists  were  used  to  observe  and  select  actions. 
This  led  to  the  discovery  that  recalculations  and  rerankings  were  performed  each  time  an  action 
list  was  selected  for  display.  The  time  required  to  perform  this  recalculation  was  approximately 
15  seconds  and  was  inefficient  since  the  calculations  and  rankings  were  performed  during  the 
initialization  of  the  diagnostic  module.  This  inefficiency  may  become  a  major  problem  when  a 
full  data  base  is  installed  and  several  symptoms  are  observed. 


Testing  and  Result  Conclusions 

The  testing  performed  on  the  IMIS  diagnostic  module  revealed  a  great  capacity  to 
overcome  many  potential  problems  that  might  be  encountered  in  demanding  scenarios.  There 
were,  however,  several  scenarios  in  which  the  IMIS  diagnostic  module  could  not  function 
normally  as  a  diagnostic  aid.  Most  of  the  scenarios  which  resulted  in  failure  of  the  diagnostic 
module  were  associated  with  the  creation  or  use  of  the  diagnostic  data  base  or  the  technical 
order  data  base.  The  detailed  test  procedures  and  results  from  each  test  performed  are  provided 
in  the  Appendix;  however,  some  of  the  most  important  test  results  are  summarized  below: 
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1.  Data  hase  accuracy  requirements.  Many  of  the  potential  errors  in  the  diagnostic  data 
base  or  the  technical  order  data  base  can  virtually  negate  the  effectiveness  of  the  diagnostic 
module.  It  is  absolutely  essential  that  these  data  bases  be  exhaustively  reviewed  for  accuracy 
and  completeness. 

2.  Technical  order  navigation.  Both  FOM  and  cannibalization  problems  resulted  in 
inadvertent  termination  of  the  diagnostic  process  before  successful  completion.  As  these 
problems  are  frequently  encountered  during  diagnostics  processes,  a  capability  to  navigate 
successfully  through  these  types  of  exercises  should  be  developed. 

3.  Criticality  treatment.  The  criticality  treatment  in  this  version  of  the  diagnostic 
module  is  almost  wholly  adopted  from  a  version  of  the  diagnostic  module  that  was  based  upon 
a  single  fault  assumption.  That  assumption  fostered  a  policy  of  "find  a  single  action  decision" 
for  the  criticality  problem.  Adopting  the  single  fault  solution  virtually  unchanged  with  the 
current  multiple  fault  processing  capability  has  resulted  in  inefficiencies  that  must  be  corrected. 

4.  "But  Not"  data  entry.  Many  efficiencies  can  be  gained  by  adopting  a  "but  not"  data 
entry  and  processing  capability.  The  diagnostic  module  cannot  take  advantage  of  these 
efficiencies.  This  shortfall  occurs  in  initializing  the  diagnostic  process,  reinitializing  after  a 
functional  check,  and  entering  the  results  of  a  MOT. 


V.  RECOMMENDATIONS 
Fault  Processing  Issues 


Can  Not  Duplicate  (CND) 

The  diagnostic  module  must  employ  a  requirement  that  fault  verification  be  completed 
prior  to  initializing  for  fault  isolation.  A  strategy  should  be  developed  to  allow  the  diagnostic 
module  to  account  for  and  act  upon  the  CND  problem. 

The  strategy  proposed  will  take  advantage  of  the  data  collected  by  the  diagnostic 
module  during  a  diagnostic  session,  the  data  stored  in  the  CAMS,  and  off-line  processing 
performed  at  the  IMIS  workstation.  An  outline  of  the  proposed  strategy  follows. 

1.  Initial  CND  occurrence  in  a  subsystem: 

Accept  this  CND  as  a  transient  problem  and  enter  CND  on  AFTO  Form  349. 

2.  Second  CND  occurrence  in  the  subsystem: 

a.  Search  for  and  perform  any  tests  which  are  not  included  in  the  fault  verification 
process,  but  span  faults  implicated  by  the  reported  symptom. 

b.  If  no  tests  are  available  that  meet  criteria  in  a.,  perform  action  at  the  top  of  the 
"Best  Action"  list. 

c.  If  tests  are  available  that  meet  criteria  of  a.  and  all  pass,  then  perform  action  at 
the  top  of  the  "Best  Action"  list. 

d.  If  tests  are  selected  that  meet  criteria  of  a.  and  fail,  then  complete  diagnostics 
normally. 

3 .  Third  and  subsequent  CND  occurrence: 

Perform  second  and  subsequent  recommended  action  on  the  "Best  Action"  list. 
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Intermittent 


Diagnosing  intermittent  faults  is  similar  to  diagnosing  CND  faults.  The  strategy 
developed  for  the  CND  problem  will  probably  handle  most  intermittent  faults.  Those  not 
caught  by  the  CND  strategy  will  be  corrected  by  their  fortuitous  exposure  during  testing.  No 
action  to  address  this  problem  by  itself  is  recommended. 


Tests  That  Fix 

These  will  be  relatively  rare,  and  they  result  in  successful  system  rectification.  The 
impact  is  a  slight  increase  in  the  RTOK  rate  as  shown  in  Test  4,  the  Appendix.  Action  on  this 
problem  should  be  deferred  until  later  development  of  IMIS  diagnostics. 


Loss  of  Test  Information 

This  problem  occurs  only  in  spanned  faults  from  a  failed  test;  therefore,  the  fault  will  be 
isolated  eventually.  The  impact  is  inefficient  diagnostics  rather  than  failed  diagnostics.  It  is 
recommended  that  a  strategy  to  overcome  this  effect  of  the  multiple  fault  handling  strategy  be 
developed. 


Access  Groups 

When  an  access  group  strategy  is  employed,  the  time  invested  in  gaining  access  to  a  test 
point  is  not  wholly  a  penalty  against  the  test  being  evaluated  if,  by  gaining  access,  other  useful 
test  points  are  also  revealed.  The  Best  Test  algorithm  in  Version  1.0  of  the  IMIS  diagnostic 
module  was  modified  to  take  advantage  of  this  strategy.  The  diagnostic  module  submitted  for 
validation  and  verification  did  not  make  use  of  this  modification. 


I'But  Not"  Data  Entry  and  Return  of  More  Than  One  Outcome  from  a  MOT 

The  "But  Not"  data  entry  can  be  beneficial  in  rapidly  reducing  the  set  of  faults  to  be 
isolated.  However,  the  usefulness  of  this  technique  depends  upon  the  assumptions  and 
requirements  of  the  test  procedures.  Tests  can  be  authored  in  at  least  three  methods: 

1 .  Exit  at  first  failure.  The  test  outcome  implicates  a  specific  set  of  faults  but  does  not 
exonerate  faults  in  other  possible  outcomes.  Tests  are  authored  in  this  manner  if  later  outcomes 
depend  upon  successful  results  from  earlier  outcomes. 

2.  Complete  test  and  enter  a  single  result.  Test  outcomes  are  interdependent,  and  the 
single  result  chosen  will  implicate  an  appropriate  set  of  faults;  other  fault  sets  are  exonerated. 

3.  Complete  test  and  enter  all  observed  failures.  Each  step  of  the  test  presents  an 
independent  result  (no  future  steps  in  the  test  depend  upon  a  successful  outcome  from  an  earlier 
step);  faults  spanned  by  the  nonobserved  outcomes  are  exonerated. 

Tests  authored  in  accordance  with  method  3  can  make  maximum  use  of  "But  Not"  data 
entry.  However,  the  current  diagnostic  module  was  designed  and  coded  in  accordance  with 
method  2. 
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We  recommend  the  diagnostic  module  and  the  appropriate  data  tables  (or  TO  data)  be 
redesigned  to  take  maximum  advantage  of  a  given  test  procedure.  Pending  such  a  redesign,  the 
current  diagnostic  module  code  should  be  modified  so  that  nonentered  symptoms  or  nonentered 
test  results  do  not  exculpate  faults  inadvertently. 

Handling  of  Maintenance  Actions 

The  diagnostic  module  treats  maintenance  actions  (actions  such  as  calibrate,  adjust, 
align,  etc.)  the  same  as  any  other  rectification.  The  result  is  a  failure  of  diagnostics.  This 
problem  can  be  readily  overcome  by  simply  questioning  the  user  as  to  the  success  of  the 
maintenance  action.  A  "not  able  to  complete”  result  will  immediately  implicate  some  other  fault 
and  make  die  need  for  a  functional  check  unnecessary. 

Criticality 

The  diagnostic  module  should  be  changed  so  the  technician  can  continue  diagnostics 
and  repair  after  die  critical  functions  have  been  exonerated.  The  basic  algorithms  around  which 
the  criticality  function  was  built  were  modeled  for  a  single  fault  assumption.  The  criticality 
function  in  the  diagnostic  module  should  be  redesigned  to  overcome  the  identified 
shortcomings. 


DataBase  Issues 


InaccurateJlncomplete  Data 

The  consequences  of  inaccurate  data  ranged  from  inefficient  diagnostic  sequences  to 
total  inability  to  complete  diagnostics  and  occasionally  to  total  system  lockup.  The  following 
are  recommended: 

1.  Allow  for  MTBF  values  greater  than  99,999  hours  in  the  data  base  authoring  form. 

2.  Include  editing  checks  in  the  data  base  to  prevent  errors  such  as 

a.  faults  with  no  rectifications, 

b.  symptoms  with  no  faults, 

c.  blank  MTBFs  and  weights, 

d.  zero  MTBFs  and  weights, 

e.  sum  of  weights  not  equal  to  100,  and 

f.  test  and  rectification  times  equal  to  zero  or  blank. 

3.  Perform  verification  of  die  fault-symptom-test-rectification  relationships  in  the 
diagnostic  model  data  base  to  ensure  that  the  system  under  test  has  been  modeled  correctly 

4.  Recode  the  diagnostic  module  software  so  that  a  missing  weight  in  an  implicated 
fault  causes  all  processing  to  revert  to  the  use  of  MTBF. 

3.  Ensure  that  the  block  diagrams  accurately  depict  the  fault  connectivity  and,  as 
closely  as  possible,  actual  system  or  functional  layout  during  verification  of  the  diagnostic  data 
base  for  a  specific  system. 
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Human  Interface  Issues 


Symptom  Entrv/Change 

Once  the  diagnostic  module  starts  fault  processing,  deletion  of  a  symptom  exculpates  all 
faults  implicated  by  that  symptom.  The  only  recovery  method  available  at  this  time  is  to  back 
up  several  steps  to  an  activity  prior  to  the  error  and  repeat  all  activities  which  have  occurred 
since  the  error.  The  diagnostic  module  should  have  a  facility  to  recover  easily  from  an 
accidental  symptom  deletion.  This  recommendation  entails  developing  a  procedure  to  go  back 
several  steps  with  a  single  keystroke  or  a  short  keystroke  sequence.  The  system  should 
recalculate  the  current  situation  based  upon  all  actions  taken  before  the  error  occurred 

Test/Functional  Check  Results 

The  diagnostic  module  should  be  given  a  facility  to  easily  correct  any  test  or  functional 
check  result  which  has  been  entered  incorrectly.  The  only  recovery  method  available  at  this 
time  is  to  back  up  several  steps  to  an  activity  prior  to  the  error  and  repeat  all  activities  which 
have  occurred  since  the  error.  The  recovery  process  should  allow  for  a  single  keystroke  or  a 
short  keystroke  sequence  to  return  to  the  place  where  the  error  occurred  and  recalculate  the 
current  position  based  on  all  actions  after  the  error. 

MIL-STD-1553  Data  Bus  Interfacing 

Although  automatic  reentry  of  fault/symptom  information  from  the  MIL-STD-1553  data 
bus  has  not  been  implemented  in  the  IMIS  diagnostic  module,  two  problem  areas  must  be 
addressed  for  correct  operation: 

1 .  A  fault/symptom  verification  routine  must  be  implemented  to  confiim  the  initial  set 
of  symptoms  from  the  AFTO  Form  349.  This  can  be  accomplished  by  utilizing  symptoms 
returned  from  operations  (OPS)  checks  and  BITs. 

2.  Symptoms  entered  from  BITs  on  the  MIL-STD-1553  data  bus  should  automatically 
update  BIT-observed  symptoms  during  a  functional  check.  Human-observed  symptoms 
should  only  be  updated  manually. 


IsgMcalQrdgr  Sequencing 


Tests  showed  that  the  diagnostic  module  becomes  totally  confused  when  one  attempts 
to  obtain  technical  order  data  for  FOMs  or  cannibalization  routines  through  the  Table  of 
Contents  opdons.  The  diagnostic  module  should  be  redesigned  to  accommodate  the  technical 
order  "calls/options"  required  to  perform  these  actions. 
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ACRONYMS 


AFTO 

- 

Air  Force  Technical  Order 

AFTOMS 

- 

Air  Force  Technical  Order  Management  System 

APS 

- 

Authoring  and  Presentation  System 

Brr 

- 

Built-In  Test 

CAMS 

- 

Core  Automated  Maintenance  System 

CND 

- 

Can  Not  Duplicate 

F(N) 

- 

Fault  (Number  for  fault  identification) 

FCC 

- 

Fire  Control  Computer 

FOM 

- 

Facilitate  Other  Maintenance 

IMIS 

- 

Integrated  Maintenance  Information  System 

MOT 

- 

Multiple  Outcome  Test 

MTBF 

- 

Mean  Time  Between  Failure 

cm 

- 

Outcome  (Number) 

OPS 

- 

Operations 

PCMAS 

- 

Portable  Computer-Based  Maintenance  Aiding  System 

R(N) 

- 

Rectification  (Number) 

R&R 

- 

Remove  and  Replace 

RT 

- 

Receiver/Transmitter 

RTOK 

- 

Retest  OK 

REMIS 

- 

Reliability  and  Maintainability  Information  System 

S(N) 

- 

Symptom  (Number) 

T(N) 

- 

Test  (Number) 

TO 

- 

Technical  Oder 

V&V 

- 

Validation  and  Verification 
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GLOSSARY 


Action.  A  diagnostic  or  corrective  procedure  performed  by  a  maintenance  technician. 


Availability.  A  component's  obtainability  for  use  in  the  diagnostics  process. 

Best  Action.  A  diagnostic  software  equation  which  chooses  the  optimum  action  from  among 
available  rectification  actions. 


Best  Test.  A  diagnostic  software  procedure  which  chooses  the  optimum  test  from  among  those 
available  at  any  point  in  the  diagnostic  sequence. 


Component.  The  lowest  physical  level  of  indenture  on  which  a  maintenance  technician  at  a 
given  level  of  maintenance  (Organizational  (O),  Intermediate  (I),  or  Depot  (D))  will 
normally  work.  For  example,  an  organizational  level  maintenance  technician  would 
consider  a  Line  Replaceable  Unit  (LRU)  as  a  component;  whereas  an  intermediate  level 
technician  would  consider  the  LRU  an  end  item  and  the  Shop  Replaceable  Unit  (SRU) 
a  component. 

Criticality.  A  measure  of  need  for  a  particular  system  capability.  For  example,  a  fault  in  an  air- 
to-ground  function  might  not  be  critical  for  an  air  defense  sortie;  whereas,  a  fault  in  an 
air-to-air  function  would  be  critical  for  the  same  sortie  requirement. 

Dominant  Action.  A  rectification  action  whose  likelihood  of  success  is  so  great  that  it  is 
recommended  prior  to  available  tests  that  would  further  reduce  the  plausible  set. 

Exculpated  Fault.  A  potential  fault  that  has  been  eliminated  from  the  set  of  possible  faults 
causing  the  problem. 

Fault.  That  which  causes  a  piece  of  equipment  to  malfunction  in  some  fashion.  A  fault  is  the 
manifestation,  through  either  inference  or  direct  observation,  of  a  failure  within  a 
system. 

Functional  Check.  A  test  performed  to  ensure  that  a  rectification  action  has  been  successful  in 
restoring  a  system  to  operational  status. 

Mean  Time  Between  Failure  (MTBFL  The  unit  of  reliability  used  in  this  program  as  a 
predictor  of  fault  likelihood.  Its  inverse  is  the  failure  rate. 


Multiple  Faults.  An  event  where  two  or  more  faults  (failed  components)  occur  simultaneously. 

Multiple  Outcome  Test  (MOT).  A  test  procedure  which  does  not  have  a  binary  pass/fail  result 
The  procedure  may  have  any  number  of  outcomes;  however,  each  is  unique  and 
distinguishable  from  all  other  outcomes. 
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Operational  fOPS~>  Check.  Frequently  used  interchangeably  with  functional  check.  In  this 
document,  it  is  a  test  of  limited  span  that  ensures  that  a  repair  action  has  been  successful 
in  removing  a  specific,  isolated  fault. 

Parent  Rectification.  A  rectification  that  includes  or  contains  other  lower  level  rectifications. 
For  example,  replacing  an  initial  navigation  platform  would  include  an  "alignment." 

Plausible  Set.  The  set  of  possible  faults  which  could  logically  be  expected  to  have  led  to  an 
observed  or  indicated  faulty  condition.  The  elements  in  this  set  of  faults  contain  single 
faults  or  combinations  of  faults  that  are  not  redundant. 

Rectification.  The  repair  of  a  fault  or  set  of  faults  which  alleviates  a  symptom  or  set  of 
symptoms. 

Repair  Time.  The  time  required  to  complete  system  repair  after  a  fault  is  isolated.  (It  may 
include  access  times  if  that  time  has  not  already  been  expended  earlier  in  the  process.) 
It  includes  the  time  required  to  reinstall  original  components  removed  unnecessarily  as 
part  of  diagnostics,  secure  and  close  all  access  panels  in  the  aircraft,  and  perform  the 
final  functional  check. 

Symptom.  An  observable  indication  that  a  malfunction  exists  within  a  piece  of  equipment 
(e.g.,  "Receiver,  no  audio,  or  MFL37"). 

Test  A  prescribed  sequence  of  actions  that  implicates  or  exonerates  a  set  of  faults. 

Test  Time.  The  time  required  to  perform  a  test.  It  includes  access  time,  time  to  gather 
necessary  test  equipment  and  tools,  time  to  conduct  the  test  procedures,  and  time 
needed  to  record/interpret  test  results. 
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APPENDIX: 

EMIS  DIAGNOSTIC  MODULE  V&V  TEST  SHEETS 
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IMIS  Diagnostic  Module  V&V  Test  Sheet 


1.  Test  No.  J. 

2.  Test  Scenario:  Can  Not  Duplicate  (CND-) 

3.  Test  Model: 
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F-Fault  S -Symptom  R-Rectification  T-Test  MOT-Multiple  Outcome  Test 


4.  Test  Procedure/Scenario:  Symptom  SO  was  reported  on  the  AFTO  Form  349,  but  the  symptom 
cannot  be  duplicated  during  die  fault  verification  process  due  to  the  CND  problem.  F2  was 
designated  as  the  true  system  fault.  Diagnostics  are  attempted  by  simply  initiating  the  diagnostic 
sequence  with  SO. 

5.  Steps: 


Action  Recommendation 


Comment 


Case  1.  Fault  verification  is  performed  prior  to  initialization. 


Functional  check  (pass)  No  faults  found. 

Case  2.  Fault  verification  is  not  performed  prior  to  initialization,  diagnostics  initialized  on  pilot 
reported  symptoms. 


a.  Initialize  w/SO&Sl 

b.  T7  (pass) 

c.  RO 

d.  Functional  check  (pass) 


T7 

RO 

Functional  check 
Failed  faults  found 


T7  passes  as  a  result  of  the 
CND. 


RO  incorrectly  identified 
as  having  "fixed"  the 
problem. 


6.  Test  Result:  The  IMIS  diagnostic  module  cannot  handle  the  CND  problem  gracefully. 
Although  the  apparent  problem  appeared  to  have  been  repaired,  in  fact,  whatever  was  causing  the 
CND  problem  is  still  in  the  system  unless  it  was  fortuitously  corrected  by  the  action  to  repair  RO. 
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IMIS  Diagnostic  Module  V&V  Test  Sheet 


1.  Test  No.  _2 

2.  Test  Scenario:  Intermittent 

3.  Test  Model: 
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F-Fault  S-Symptom  R-Rectification  T-Test  MOT-Multiple  Outcome  Test 


4.  Test  Procedure/Scenario:  Symptoms  SO  and  S 1  were  reported  on  the  AFTO  Form  349,  FO  is 
designated  as  the  intermittent  fault,  and  F4  is  present  as  a  normal  fault.  FO  is  present  during  fault 
verification,  but  comes  and  goes  randomly  during  fault  isolation  tests. 

5.  Steps: 


Action 

Recommendation 

Comment 

a.  Initiate  w/S0&  SI 

T2 

T2  was  not  the  diagnostic 
module's  recommended  action 
but  was  chosen  to 
demonstrate  the  occurrence  of 
an  intermittent  fault 

b.  T2  (fail) 

TO 

F0  and  F3  implicated.  FO 
active  during  this  test. 

c.  TO  (pass) 

R2 

FO  disappears  randomly 
during  this  test;  therefore,  F3 
is  implicated. 

d.  R2 

Functional  check 

e.  Functional  check  (fail) 

T7 

This  check  fails;  however, 
only  SI  returns  from  this 
check.  F0,F1,F2  are 
exculpated. 

f.  T7  (pass) 

R4 

F1,F2,F3,  all  OK.  F4 
sole  remaining  fault 
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g.  R4  Functional  check 

h.  Functional  check 

1.  Pass  Failed  faults  found  If  FO  remains  dormant, 

this  will  be  die  result. 

F2  and  F4  identified  as 
having  caused  the  problem. 

F2  identification  incorrect 
FO  problem  still  not  identified. 

2.  Fail  (SO  returned)  Failed  faults  not  found  All  faults  associated  with  SO 

have  been  exculpated  and 
diagnostics  end  without  the 
rectification  of  FO. 

6.  Test  Result:  The  IMIS  diagnostic  module  cannot  handle  the  "intermittent"  problem  gracefully. 
Although  the  apparent  problem  sometimes  appears  to  have  been  repaired,  in  fact,  the  intermittent 
problem  is  still  in  the  system  at  FO. 
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IMIS  Diagnostic  Module  V&V  Test  Sheet 


1.  Test  No.  3. 

2.  Test  Scenario:  Intermittent 

3.  Test  Model: 
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F-Fault  S-Symptom  R-Rectification  T-Test  MOT-Multiple  Outcome  Test 


4.  Test  Procedure/Scenario:  Symptom  SO  was  reported  on  the  AFTO  Form  349,  FO  is  the 
intermittent  fault  which  has  resulted  in  the  presence  of  SO.  FO  is  present  during  fault  verification, 
but  comes  and  goes  randomly  during  fault  isolation  tests. 

5.  Steps: 

Action  Recommendation  Comment 


a.  Initiate  w/SO  T5 

b.  T5  (pass)  T4  F20K 


c.  T4  (pass)  R1 

d.  R1  Functional  check 

e.  Functional  check  (pass)  Failed  faults  found 


FO  went  away  during 
testing;  so  T4  passes, 

FO  exculpated,  FI  sole 
remaining  member  of  the 
Plausible  Set 


This  test  passes  because  FO  is 
still  dormant,  R1  incorrectly 
identified  as  having  "fixed" 
the  problem. 


6.  Test  Result:  The  IMIS  diagnostic  module  cannot  handle  the  "intermittent"  problem  gracefully. 
Although  the  apparent  problem  appeared  to  have  been  repaired,  in  fact  whatever  was  causing  the 
intermittent  problem  is  still  in  the  system  unless  it  was  fortuitously  captured  by  the  action  to  repair 
Rl. 
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1.  Test  No.  A 

2.  Test  Scenario:  Tests  That  Fix 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptom  SO  was  repented  on  the  AFTO  Form  349,  and  FO  was 
designated  as  the  true  system  fault  T4  that  spans  FO  will  be  considered  to  remove  the  fault. 

5.  Steps: 

Action  Recommendation  Comment 

a.  Initialize  w/SO  T4  T4  was  designated  as  a  test 

that  fixes. 


b.  T4  (passes  and  fixes  FO)  R2 

c.  R2  Functional  check 

d.  Functional  check  (pass)  Failed  faults  found 


FO  is  fixed  by  performing  T4, 
but  diagnostics  continue 
choosing  the  next  highest 
ranked  action  R2. 


Diagnostics  end  reporting  that 
R2  fixed  the  fault  when  T4 
fixed  the  true  fault  FO. 


6.  Test  Result:  The  IMIS  diagnostic  module  does  not  recognize  that  tests  can  fix  faults.  The 
problem  appeared  to  have  been  repaired  by  the  performance  of  R2,  but  T4  fixed  the  true  fault,  FO. 
R2  is  also  an  unnecessary  action  considering  that  FO  was  fixed  by  the  performance  of  T4. 


IMIS  Diagnostic  Module  V&V  Test  Sheet 


1.  Test  No. 

2.  Test  Scenario:  Loss  of  Test  Information 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptoms  SO  and  SI  were  reported  on  the  AFTO  Form  349  while 
F2  and  F3  were  designated  as  the  tree  system  faults. 

5.  Steps: 

Action  Recommendation  Comment 

a.  Initialize  w/SO&Sl  T2  T2  is  not  the  diagnostic 

module's  recommended 
action,  but  was  chosen  to 
demonstrate  loss  of  test 
information. 


b.  T2  (fail) 

TO 

T2  fails  implicating  F0  and/or 
F3. 

c.  TO  (pass) 

R2 

TO  passes  exculpating  F0  and 
FI.  At  this  point  F3  and  F2 
are  known  faults  and  both 
should  be  rectified. 

d.  R2 

Functional  check 

e.  Functional  check  (fail) 

T7 

f.  T7 

(Outcome  1.3  observed) 

R3 

T7  splits  F3  from  F4,  but  is  a 
useless  test  because  F3  is  a 
known  fault  from  previous 
test  results. 

g.  R3 


Functional  check 
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h.  Functional  check  (pass)  Failed  faults  found 


Loss  of  test  information 
resulted  in  the  performance  of 
T7  (useless  test)  and 
inefficient  diagnostics. 

6.  Test  Result:  The  IMIS  Diagnostic  Module  can  lose  test  information  during  diagnostic 
iterations.  Information  gained  from  the  performance  of  T2  and  TO  isolated  F3,  but  diagnostics 
continued  to  attempt  fault  isolation  by  splitting  faults  F3  and  F4  with  T7. 
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1.  Test  No.  _£ 

2.  Test  Scenario:  "But  Not "  Data  Entry 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptom  SO  was  reported  on  the  AFTO  Form  349,  but  not  SI;  and 
FI  was  designated  as  the  true  system  fault.  The  occurrence  of  one  symptom  but  not  another 
symptom  should  result  in  the  exculpation  of  faults  that  are  dependent  on  the  occurrence  of  both 


symptoms. 

5.  Steps: 

Action 

Recommendation 

Comment 

a.  Initialize  w/SO 

R2 

F0,  FI,  and  F2  were  selected 

b.  R2 

c.  Functional  check  (fail) 

d.  R1 

e.  Functional  check  (pass) 

Functional  check 

R1 

Functional  check 

Failed  faults  found 

by  the  diagnostic  module  as 
plausible  faults.  F2  should 
not  be  a  plausible  fault 
because  it  is  dependent  on  the 
occurrence  of  both  SO  &  SI. 

6.  Test  Result:  The  IMIS  diagnostic  module  presently  does  not  handle  "But  Not"  data  entries,  and 
as  a  result,  completes  diagnostic  sessions  inefficiently  by  including  faults  that  are  dependent  on  the 
occurrence  of  symptoms  that  are  not  observed. 
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1.  Test  No.  _2 

2.  Test  Scenario:  Handling  of  Maintenance  Actions 

3.  Test  Model: 
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4.  Test  Proccdure/Scenario:  Symptom  SO  was  reported  on  the  AFTO  Form  349,  and  FI  was 
designated  as  the  true  system  fault  and  R1  a  maintenance  action.  A  maintenance  action  may  be 
recommended  by  the  diagnostic  module,  but  while  performing  this  action,  the  maintenance 
technician  discovers  that  it  cannot  be  completed  and  wishes  to  continue  diagnostics. 

3.  Steps: 


Action  Recommendation 


Comment 


a.  Initialize  w/SO  T7 

b.  T7  R1 

(Outcome  1.1  observed) 

c.  R1  Functional  check 


d.  Functional  check  (fail)  RO 


e.  RO  Functional  check 

f.  Functional  check  (fail)  T7 

g.  T7  Repeat  of  steps  a.  -  f. 


FI  is  isolated. 


Rl,  a  maintenance  action, 
cannot  be  performed 
successfully,  thus  implicating 
the  parent  rectification,  R5. 

The  fail  on  the  functional 
check  is  a  result  of  the  non¬ 
performance  of  Rl. 
Diagnostics  can  be  completed 
with  RS,  but  the  diagnostic 
module  assumes  that  Rl  has 
been  performed  successfully 
and  FI  has  been  repaired. 
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6.  Test  Result:  The  IMIS  diagnostic  module  does  not  recognize  that  R1  could  not  be  performed.  It 
continues  diagnostics  as  if  R1  had  been  completed  and  assumes  that  the  fault  lies  elsewhere  in  the 
system.  Although  R5  is  a  viable  rectification,  it  is  never  recommended. 
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1.  Test  No.  _£ 

2.  Test  Scenario:  Criticality 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptom  SI  was  reported  on  the  AFTO  Form  349,  F3  was 
designated  as  critical  and  F4  as  the  true  system  fault  Designating  faults  as  critical  for  the  next 
sortie  causes  the  diagnostic  module  to  attack  critical  faults  first,  where  performing  normal 
diagnostics  may  result  in  a  quicker  total  system  rectification  time. 

5.  Steps: 


Action 

Recommendation 

Comment 

a.  Initialize  w/S  1 

R3 

The  diagnostic  module 

attempts  to  rectify  critical 

faults  first. 

b.  R3 

Functional  check 

c.  Functional  check  (fail) 

R2 

d.  R2 

Functional  check 

e.  Functional  check  (fail) 

R4 

f.  R4 

Functional  check 

g.  Functional  check  (pass) 

Failed  faults  found 

If  criticality  had  not  been 

invoked,  T7  would  have  been 
performed  and  passed, 
exculpating  F2  and  F3, 
and  R4  would  have  been  the 
next  recommended  action. 


6.  Test  Result:  The  IMIS  diagnostic  module's  criticality  function  can  be  a  useful  tool,  but  the 
module's  performance  can  be  hindered  if  the  true  fault  does  not  reside  in  a  mission  critical 
component 
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1.  Test  No.  _2 

2.  Test  Scenario:  Unseen  Fault 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptom  SO  was  reported  on  the  AFTO  Form  349,  and  a  fault 
weight  of  0  was  assigned  to  FI  (the  true  fault).  Initial  data  collection  and  entry  may  result  in  a 
fault/symptom  weight  assignment  of  0  as  a  default  value  if  no  weight  has  been  given. 

5.  Steps: 

Action  Recommendation  Comment 


a.  Initialize  w/SO  and  an  R2 
FI  weight  value  of  0. 


b.  R2 

c.  Functional  check  (fail) 

d.  R0 

e.  Functional  check  (fail) 

f.  R2 


Functional  check 
R0 

Functional  check 
R2 

Repeat  of  steps  b.-f. 


FI  was  displayed  in  the 
fault/symptom  mapping  table, 
but  tests  and  rectifications 
to  isolate  and  repair  FI  were 
not  displayed  in  the  ranking 
lists. 


6.  Test  Result:  The  IMIS  diagnostic  module  recognizes  a  fault/symptom  weight  value  of  zero,  but 
considers  the  combination  an  infeasible  alternative  for  isolation  and  repair. 
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1.  Test  No.  _1Q 

2.  Test  Scenario:  Useless  Fault 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptom  SI  was  reported  on  the  AFTO  Form  349,  and  FO  was 
incorrectly  modeled  in  the  Table  Maker  as  an  intersecting  fault.  F3  was  designated  as  the  true 
system  fault  Incorrect  modeling  of  fault/symptom  relationships  may  result  in  the  inclusion  of 
faults  in  the  diagnostic  process  which  are  unrelated  to  the  observed  symptom. 

5.  Steps: 


Action 


Recommendation 


Comment 


a.  Initialized  w/S  1 


T4 


b.  T4(pass) 

c.  R5 

d.  Functional  check  (pass) 


R5 

Functional  check 
Failed  faults  found 


T4  was  not  the  diagnostic 
module's  recommended  action 
but  was  chosen  to 
demonstrate  a  useless  fault 


6.  Test  Result:  The  IMIS  diagnostic  module  does  not  recognize  an  incorrectly  modeled 
fault/symptom  relationship  and  performs  diagnostics  with  the  given  incorrect  information  resulting 
in  an  increased  time  to  repair  the  aircraft. 
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1.  Test  No.  JLi 

2.  Test  Scenario:  Incorrect  Weights/Weights  Do  Not  Sum  to  100 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptom  SI  was  reported  on  the  AFTO  Form  349  with  F2  as  the 
true  system  fault.  Fault/symptom  probability  errors  occur  when  faults  are  implicated  by  a 
symptom  and  included  in  the  fault/symptom  supporting  data  base,  but  carry  incorrect  weight  values 
or  the  weights  do  not  sum  to  100. 

5.  Steps: 

Action 

a.  Initialize  w/Sl 

b.  R4 

c.  Functional  check  (fail) 

d.  R2 

e.  Functional  check  (pass) 


Recommendation  Comment 

R4 

Functional  check 
R2 

Functional  check 
Failed  faults  found 


6.  Test  Result:  The  IMIS  diagnostic  module  does  not  recognize  incorrect  weight  values  or 
weights  that  do  not  sum  to  100,  and  diagnostics  continue  normally  with  respect  to  the  weights 
assigned. 
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1.  Test  No.  J2 

2.  Test  Scenario:  Incorrect  MTBFs  (MTBF  =  infinity) 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptom  SO  was  reported  on  the  AFTO  Form  349,  and  FI  was 
designated  as  the  true  fault  The  largest  value  that  could  be  input  for  an  MTBF  was  9999.  If  an 
MTBF  value  is  very  high,  is  information  lost  during  diagnostic  calculation  for  fault/symptom 


weights? 

5.  Steps: 

Action 

Recommendation 

Comment 

a.  Initialize  w/SO 

R1  in  the  options  list 
displayed  0  (rounded),  but 
the  actual  value  observed  in 

dbxTM  was  .000499. 

6.  Test  Result:  The  IMIS  diagnostic  module  lost  no  information  as  a  result  of  rounding  off 
numbers  for  the  option- list  display,  and  all  tests  and  rectifications  were  displayed  in  the  appropriate 
ranking  with  respect  to  their  MTBF  values. 
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1.  Test  No.  _I2 

2.  Test  Scenario:  Incorrect  MTBFs  (MTBF  =  zero) 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptom  SO  was  reported  on  the  AFTO  Form  349,  and  FO  was 
designated  as  the  true  fault.  Fault  MTBF  values  are  accessed  in  the  event  that  fault  weights  are  not 
available.  During  data  entry  an  MTBF  value  may  not  be  available  and  be  assigned  0  for  a  default 


value. 

5.  Steps: 

Action 

Recommendation 

Comment 

a. 

Initialize  w/SO 

R1 

Only  tests  and  rectifications 
for  FI  were  recommended. 

R1  and  R5  had  option-list 
probabilities  of  214%. 

b. 

R1 

Functional  check 

c. 

Functional  check  (fail) 

R1 

d. 

R1 

Repeat  steps  b.-d. 

6.  Test  Result:  The  IMIS  diagnostic  module  cannot  handle  an  MTBF  of  0.  Dbx™  denoted 
processing  errors  had  occurred,  and  all  other  faults,  except  FI,  were  alleviated  from  consideration. 
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1.  Test  No.  _14 

2.  Test  Scenario:  Unassiened  Fault  Weight/MTBF  Values 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptom  SO  was  reported  on  the  AFTO  Form  349,  and  FI 
designated  as  the  true  system  fault,  was  not  assigned  any  weight  value.  During  data  entry  within 


the  data  base,  some  faults  may  not  be 
5.  Steps: 

assigned  a  weight  value. 

Action 

Recommendation 

Comment 

a.  Initialize  w/SO 

R1 

The  value  of  65535  was 
received  by  the  diagnostic 
module  and  diagnostics 
continue  by  accessing  MTBF 
values  for  FI. 

b.  R1 

Functional  check 

c.  Functional  check  (pass) 

Failed  faults  found 

6.  Test  Result:  The  IMIS  diagnostic  module  was  designed  to  use  fault  weights.  In  the  event  that 
fault  weights  are  not  available,  MTBF  values  are  used  as  defaults.  Diagnostics  were  completed 
successfully  relative  to  the  MTBF  values  assigned.  The  only  instance  that  would  create  diagnostic 
difficulties  would  be  if  there  were  missing  MTBF  values  or  MTBFs  of  0.  The  Table  Maker 
requires  a  positive  integer  or  zero  value  for  an  MTBF  entry;  therefore,  undefined  entries  in  the  data 
tables  cannot  exist  and  do  not  pose  diagnostic  difficulties. 
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1.  Test  No.  Ji 

2.  Test  Scenario:  Unmodeled  Symptom  (Manual  Entry') 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptom  S2  is  observed  by  a  maintenance  technician;  and  manual 
entry  of  symptom  selection  is  required  to  designate  symptom  occurrence.  S2  is  not  modeled 
within  the  diagnostic  data  base. 

5.  Steps: 


Action  Recommendation 


Comment 


S2  observed  Data  base  is  not  modeled  for 

this  symptom  and  is  not 
displayed  to  the  technician  as 
a  choice  to  initialize 
diagnostics. 


6.  Test  Result:  The  IMIS  Diagnostic  Module  cannot  be  initialized  with  a  symptom  that  is  not 
modeled  in  the  diagnostic  data  base. 
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1.  Test  No.  16 

2.  Test  Scenario:  Unmodeled  Symptom  (Automatic  Entry) 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  The  AFTO  Form  349  in  the  Authoring  and  Presentation  System  was 
modified  to  include  SO,  SI,  and  an  unknown  S2.  This  modification  simulates  a  situation  that 
occurs  when  the  downloading  from  die  1553  bus  exhibits  a  symptom  which  has  not  been  included 
in  the  diagnostic  system's  data  base. 

5.  Steps: 

Action  Recommendation  Comment 

Initialize  w/SO,  SI  &  S2  Upon  examination  of  the 

symptom  availability  list,  S2 
was  not  available,  but  SO  and 
SI  were  included  as  the 
observed  symptoms. 

6.  Test  Result:  The  IMIS  diagnostic  module  will  not  initialize  a  symptom  that  has  not  been 
modeled  in  the  diagnostic  data  base. 
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1.  Test  No.  _LZ 

2.  Test  Scenario:  Unmodeled  Weights/MTBFs  for  Fault/Svmptom  Relationship 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptom  SO  was  reported  on  the  AFTO  Form  349,  and  FI  was 
assigned  a  0  value  for  its  MTBF  and  an  unassigned  value  for  its  weight.  Faults  may  be  implicated 
by  a  symptom,  but  upon  evaluation  of  weight  and  MTBF  table  values,  there  are  missing  MTBF 
and  weight  values  for  the  same  fault. 

5.  Steps: 

Action  Recommendation  Comment 

Initialize  w/SO  System  "bombed"  Processing  errors  occurred 

and  diagnostics  were  not 
completed. 

6.  Test  Result:  The  IMIS  diagnostic  module  could  not  be  initialized  with  the  occurrence  of  an 
unmodeled  weight  and  MTBF  for  the  same  fault/symptom  relationship. 
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1.  Test  No.  18 

2.  Test  Scenario: 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptom  SO  was  reported  on  the  AFTO  Form  349.  FO  was 
assigned  a  blank  weight  value  and  F2  a  0  for  die  MTBF  value.  Faults  may  be  implicated  by  a 
symptom,  but  upon  evaluation  of  weight  and  MTBF  table  values,  there  are  missing  MTBF  and 
weight  values  for  the  different  faults. 

5.  Steps: 

Action  Recommendation  Comment 


Initialize  w/SO  The  diagnostic  module  utilized 

the  MTBF  value  for  FO  and 
the  weight  value  for  F2. 
Diagnostics  continued 
undisturbed. 

6.  Test  Result:  The  IMIS  diagnostic  module  was  able  to  accomplish  diagnostics  by  normalizing 
MTBF  values  for  FI  and  using  the  given  weight  value  for  F2.  This  occurrence  of  missing  data 
provided  no  problems  for  die  diagnostic  module. 
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1.  Test  No.  _12 

2.  Test  Scenario:  Unmodeled  Tests  and  Rectifications 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptom  SO  was  reported  on  the  AFTO  Form  349,  and  FI  was 
designated  as  the  true  system  fault.  An  observed  symptom  implicates  a  fault  few  which  no  tests  or 
rectifications  are  available. 

5.  Steps: 

Action  Recommendation  Comment 


a.  Initialize  w/SO 

b.  R2 

c.  Functional  check  (fail) 

d.  RO 

e.  Functional  check  (fail) 

f.  R2 


R2 

Functional  check 
RO 

Functional  check 
R2 


System  errors  occurred  and 
the  diagnostic  module  'locked 
up." 


6.  Test  Result:  The  IMIS  diagnostic  module  does  not  rectify  a  fault  with  no  tests  or  rectifications 
mapped  to  it.  System  errors  occurred  and  there  was  no  chance  to  recover. 
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1.  Test  No.  20 

2.  Test  Scenario:  Unassigned  Values  for  Test  and  Rectification  Times 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptom  SO  was  reported  on  the  AFTO  Form  349,  and  T7  (MOT) 
and  R2  were  assigned  blank  action  times  within  the  technical  order  authoring  system.  FO  was 


assigned  the  true  system  fault 

S.  Steps: 

Action 

Recommendation 

Comment 

a.  Initialize  w/SO 

R1 

T7  and  R2  times  were 
corrupted.  Both  actions  were 
ranked  last  on  the  options  list 
and  their  displayed  test  values 
corresponded  to  their  ranking. 
Test  time  values  displayed  in 
dbx™  were  16,776,440. 

For  example,  a  ranking  of  7 
would  display  an  action  time 
of  71.  Action  times  from 
dbx^M. 

b.  R1 

Functional  check 

c.  Functional  check  (fail) 

R0 

d.  R0 

Functional  check 

e.  Functional  check  (pass) 

Failed  faults  found 

6.  Test  Result:  Hie  IMIS  diagnostic  module  could  not  handle  blank  action  times,  and  information 
from  ranked  options  lists  as  corrupted.  Options  with  blank  action  times  were  ranked  last  on  the 
options  lists,  and  their  action  times  corresponded  to  their  ranking. 
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1.  Test  No.  _21 

2.  Test  Scenario:  Test  and  Rectification  Times  of  Zero 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptoms  SO  and  SI  were  reported  on  the  AFTO  Form  349,  and  T3 
and  RO  were  assigned  zero  action  times  within  the  technical  order  authoring  system.  FO  and  F3 


were  designated  as  true  system  faults. 

5.  Steps: 

Action 

Recommendation 

Comment 

a.  Initialize  w/SO 

T7 

T3  and  R0  times  were 
corrupted.  Both  actions  were 
ranked  last  on  the  options  list, 
and  their  displayed  test  values 
corresponded  to  their  ranking. 
Test  time  values  displayed  in 
dbxTM  Wcre  16,776,440. 

For  examples  ranking  of  7 
would  display  an  action  time 
of  71.  Action  times  from 
dbx™. 

b.  T7  (outcome  1.3) 

R3 

c.  R3 

Functional  check 

d.  Functional  check  (fail) 

R0 

S 1  alleviated  as  a  result  of  R3. 

e.  R0 

Functional  check 

f.  Functional  check  (pass) 

Failed  faults  found 

6.  Test  Result:  Hie  IMIS  diagnostic  module  could  not  handle  zero  action  times,  and  information 
from  ranked  options  lists  were  corrupted.  Options  with  zero  action  times  were  ranked  last  on  the 
options  lists,  and  their  action  times  conesponded  to  their  ranking. 
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1.  Test  No.  2L 

2.  Test  Scenario:  Test  and  Rectification  Times  of  Infinite 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptom  SO  was  reported  on  the  AFTO  Form  349,  and  T1  and  R1 
were  assigned  infinite  (99,999,999  largest  entry)  action  times  within  the  technical  order  authoring 
system.  FO  was  assigned  the  true  system  fault. 

5.  Steps: 

Action  Recommendation  Comment 


a.  Initialize  w/SO  R2 


b.  R2 

c.  Functional  check  (fail) 

d.  RO 

e.  Functional  check  (paw) 


Functional  check 
RO 

Functional  check 
Failed  faults  found 


T1  and  R1  times  were  not 
corrupted.  Both  actions  were 
ranked  last  on  the  options  list 
as  expected  from  then- 
action  times. 


6.  Test  Result:  The  IMIS  diagnostic  module  did  not  have  problems  with  infinite  action  times. 
Action  times  were  ranked  correctly  according  to  their  probability  of  occurrence  and  assigned  action 
times. 
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1.  Test  No.  _22 

2.  Test  Scenario:  Useless  Tests  and  Rectifications 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptom  SO  was  reported  on  the  AFTO  Form  349  and  FO  was 
designated  as  the  true  system  fault  R1  was  incorrectly  modeled  to  fix  FO. 

5.  Steps: 


Action 


Recommendation 


Comment 


a.  Initialize  w/SO  T4 


b.  T4  (fail) 

c.  R5 

d.  Functional  check  (fail) 

e.  R1 

f.  Functional  check  (fail) 
g-  R5 


R5 

Functional  check 
R1 

Functional  check 
R5 

Repeat  steps  c.-g. 


T4  was  not  the  diagnostics 
recommended  action  but  was 
chosen  to  show  the  results  of 
a  useless  test. 


6.  Test  Result:  The  IMIS  diagnostic  module  could  not  repair  a  fault  that  corresponded  to  a  useless 
rectification  or  test  The  diagnostic  module  receives  incorrect  diagnostic  information  from  the  tests 
and  rectifications  that  are  incorrectly  modeled.  There  are  two  outcomes  that  can  transpire  as  a 
result  of  a  useless  action: 


a.  The  fault  is  not  rectified  and  diagnostics  continue  in  an  infinite  loop. 

b.  The  fault  is  rectified  by  a  parent  rectification. 
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1.  Test  No.  JA 

2.  Test  Scenario:  Unseen  Test  Faults 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptom  SO  and  SI  were  reported  on  the  AFTO  Form  349,  and  FO 
and  F4  were  designated  as  the  true  system  fault  A  data  base  error  has  resulted  in  failure  to 


correctly  map  F4  to  T5. 

S.  Steps: 

Action 

Recommendation 

Comment 

a.  Initialize  w/S0&  SI 

T5 

T5  was  not  recommended  by 
the  diagnostic  module  but  was 
selected  to  demonstrate  an 
unseen  test  fault. 

b.  T5  (fail) 

R2 

T5  failed  due  to  F4,  but  the 
module  incorrectly  ascribes 
the  failure  to  F2  because  of 
the  data  base  error. 

c.  R2 

Functional  check 

d.  Functional  check  (fail) 

T7 

e.  T7  (pass) 

R0 

f.  R0 

Functional  check 

g.  Functional  check  (fail) 

R4 

h.  R4 

Functional  check 

i.  Functional  check  (pass) 

Failed  faults  found 

Diagnostics  end. 

6.  Test  Result:  The  IMIS  diagnostic  module  exhibits  some  inefficiencies  as  a  result  of  the  data 
entry,  but  diagnostics  were  completed  and  the  correct  rectification  was  completed  successfully. 


IMIS  Diagnostic  Module  V&V  Test  Sheet 


1.  Test  No.  21 

2.  Test  Scenario:  Missing  Technical  Order  (TO)  Information 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptom  SO  was  reported  on  the  AFTO  Form  349.  The  TO 
information  for  RO  was  remapped  to  Rl,  and  the  TO  call  for  R2  was  alleviated. 


5.  Steps: 

Action 

Recommendation 

Comment 

a.  Initialize  w/SO 

Rl 

The  following  rectification 
choices  were  not  those 
recommended  by  the 
diagnostic  module  but 
were  selected  to  demonstrate 
the  diagnostic  scenario. 

Rl  displayed  the  TO 
information  of  R0. 

b.  Back  up  out  of  Rl 

R2 

System  errors  occur  and  the 
diagnostic  module  "bombs." 

6.  Test  Result:  If  a  request  for  TO  information  produces  the  incorrect  TO  or  if  TO  information  is 
missing,  the  IMIS  diagnostic  module  does  not  have  the  ability  to  recover.  Incorrect  TOs  display  an 
incorrect  maintenance  procedure  for  a  given  test  or  rectification.  In  the  event  of  missing  TO 
information,  diagnostics  "bomb"  and  cannot  be  continued. 
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1.  Test  No.  26 

2.  Test  Scenario:  Incorrect  Criticality  Input 

3.  Test  Model: 


MOT 


WKM 

tm 

TO  T1  T2  T3 

T4  T5  T6  T7 

R0  R1  R2  R3  R4  R5  | 

MM 

iif  n 

0 

1 

1 

1 

0 

1 

0 

F 

0 

1 

FI 

■  IK1  Cl 

0 

1 

0 

0 

0 

0 

0 

u 

1.1 

1 

1 

0 

0 

0 

0 

0 

1 

N 

1.2 

1 

1 

F3 

0 

■fit  8 

0 

1 

1 

1 

0 

0 

c 

1.3 

1 

1 

F4 

0 

0 

1 

0 

0 

0 

0 

K 

0 

1 

TIME  (MIN) 

10 

10 

10 

10 

10  10 

10 

8 

10 

5 

5 

5  10 

10 

F-Fault  S-Symptom  R-Recdfication  T-Test  MOT-Multiple  Outcome  Test 


4.  Test  Procedure/Scenario:  Symptoms  SO  and  S 1  were  reported  on  the  AFTO  Form  349,  and  F2 
was  the  designated  true  system  fault.  In  the  event  that  criticality  is  incorrectly  selected,  adverse 
effects  on  diagnostics  can  occur. 

5.  Steps: 

Action  Recommendation  Comment 


a.  Initialize  w/SO  &  S 1 

T1 

Critical  test 

b.  T1  (pass) 

Failed  faults  found 

Diagnostics  end  without 
rectifying  F2,  the  true  fault. 

6.  Test  Result:  The  IMIS  diagnostic  module  ends  diagnostics  upon  exculpation  of  all  critical  faults 
and  cannot  continue  diagnostics  if  the  true  system  fault  is  not  critical.  Diagnostic  information 
learned  during  the  first  session  cannot  be  used  to  reinitialize  diagnostics  if  time  remains  to  rectify 
the  true  system  fault 
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1.  Test  No.  _2Z 

2.  Test  Scenario:  Incorrect  Symptom  Input 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptoms  SO  and  SI  were  observed.  Symptom  SO  was  reported  on 
the  AFTO  Form  349  and  not  S 1 .  FO  and  F3  were  the  designated  true  system  faults. 

5.  Steps: 


Action 


Recommendation 


Comment 


a.  Initialize  w/SO  17 

b.  T7  (outcome  1.3) 


c.  Selects  1/deselect  SO  R3 

d.  R3  Functional  check 


e.  Functional  check  (pass)  Failed  faults  found 


Realization  that  symptom  SI 
was  not  selected  during 
initialization.  Upon 
reselection  of  SI,  SO  was 
accidentally  deselected. 


6.  Test  Result:  The  IMIS  diagnostic  module  will  allow  symptom  entry  at  any  point  in  diagnostics; 
but  the  deselection  of  a  symptom  exculpates  the  faults  associated  only  with  that  symptom,  and  the 
faults  cannot  be  recovered  with  the  backup  function  or  by  reselecting  that  symptom. 
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1.  Test  No.  28 

2.  Test  Scenario:  Single  and  Continuous  Incorrect  Test  Result  Entries  and  Continuing  Diagnostics 
without  Correct  Reentry  of  Test  Outcome 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptoms  SO  and  SI  were  reported  on  the  AFTO  Form  349,  and  F2 
was  designated  as  the  true  system  fault 

5.  Steps: 


Action 

Recommendation 

Comment 

a. 

Initialize  w/SO  &  SI 

T7 

b. 

T7  (pass) 

R0 

Outcome  1.2  was  observed 
but  was  not  entered  as  a 

"fail."  The  incorrect  "pass" 
entry  on  T7  exculpates  FI, 
F2,  and  F3. 

c. 

R0 

Functional  check 

d. 

Functional  check  (fail) 

R0 

e. 

R0 

Repeat  steps  b.-e. 

F2  is  never  rectified. 

6.  Test  Result:  The  IMIS  diagnostic  module  cannot  handle  incorrect  test  result  entries  if  the  entries 
are  not  corrected.  If  the  error  is  caught  by  the  technician,  the  backup  function  can  be  used  to  back 
up  and  change  the  incorrect  test  entry,  but  if  not  caught,  can  be  devastating  to  diagnostic 
effectiveness.  Continuous  incorrect  test  entries  were  performed  with  the  same  testing  and 
produced  the  same  results. 
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1.  Test  No.  29 

2.  Test  Scenario:  Single  and  Continuous  Incorrect  Entry  from  a  Functional  Check 

3.  Test  Model: 
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4.  Test  Procedure/Scenario:  Symptoms  SO  and  SI  were  reported  on  the  AFTO  Form  349,  and  F2 
was  designated  as  the  true  system  fault, 

5.  Steps: 


Action  Recommendation 


Comment 


Case  1  ("Pass"  when  results  report  "fail") 

a.  Initialize  w/SO  &  S 1  R1  R1  was  not  the  recommended 

option  but  was  chosen  to 
demonstrate  the  results  of 
incorrect  functional  check 
result  entries. 

b.  R1  Functional  check 

c.  Functional  check  (pass)  Failed  faults  found  The  functional  check  was 

incorrectly  reported  as  passing 
because  F2  was  the  true  fault 
and  had  not  been  rectified. 

No  available  option  can 
change  the  incorrect  entry 
after  diagnostics  end. 


Case  2  ("Fail"  when  results  report  "pass") 

a.  Initialize  w/SO  &  SI  T7 

b.  T7  (outcome  1.2)  R2 

c.  R2  Functional  check 
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d.  Functional  check  (fail)  R4 


The  functional  check  was 
incorrectly  reported  as  failing 
because  F2  was  the  true  fault 
and  had  been  rectified. 


e.  R4 

Functional  check 

Single  incorrect  entry 

f.  Functional  check  (pass) 

Failed  faults  found 

Continuous  incorrect  entry 

g.  Functional  check  (fail) 

RO 

h.  RO 

Functional  check 

i.  Functional  check  (fail) 

R2 

j.  R2 

Functional  check 

k.  Functional  check  (fail) 

R4 

1.  R4 

Repeat  steps  e.-h. 

6.  Test  Result:  The  IMIS  diagnostic  module  cannot  recover  from  an  incorrect  pass  on  a  functional 
check.  A  single  incorrect  functional  check  fail  can  be  corrected  with  the  backup  function,  paging 
through  the  rectification,  and  reentering  the  functional  check  outcome,  or  will  be  corrected  when 
the  next  functional  check  is  performed.  Continuous  incorrect  functional  check  "fails"  result  in  an 
infinite  loop,  and  the  diagnostic  module  is  not  exited  although  the  true  fault  is  rectified. 
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IMIS  Diagnostic  Module  V&V  Test  Sheet 


1.  Test  No.  _2Q 

2.  Test  Scenario:  Reentry  of  Fault/Svmptom  Information  After  a  Functional  Check 

3.  Scenario/Test  Result:  Although  the  capability  for  the  IMIS  portable  computer  to  interface  with 
the  1553  bus  has  yet  to  be  implemented  for  diagnostic  initialization  and  fault  verification,  instances 
can  be  theorized  to  depict  problems  that  may  occur  when  the  results  of  a  functional  check  are 
automatically  entered  into  the  diagnostic  module. 

a.  When  automatic  reentry  of  symptoms  occur,  the  reinitialization  of  diagnostics  may 
exclude  symptoms  previously  observed  that  were  not  included  in  the  functional  check.  The 
automatic  reentry  could  also  exclude  manual  symptom  entries  which,  when  not  discovered  by  the 
maintenance  technician,  will  go  unnoticed.  In  either  case,  rectification  of  all  faults  may  not  be 
completed. 

b.  A  corrupted  symptom  code  can  appear  as  a  wrong  symptom  or  go  unnoticed. 
Symptoms  with  corrupted  symptom  codes  (that  correspond  to  other  real  symptom  codes)  will 
utilize  all  real  symptom  mapping  information,  and  diagnostics  will  attempt  to  isolate  or  repair  a 
fault  in  the  real  symptom.  Symptoms  with  corrupted  symptom  codes  (matches  are  not  made  to 
modeled  symptom  codes)  will  not  be  initialized  into  diagnostics  as  a  result  of  an  unmodeled 
symptom. 
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1.  Test  No.  _21 

2.  Test  Scenario:  Return  of  More  Than  One  Outcome  from  an  MOT 

3.  Test  Model: 
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4.  Test  Pnocedure/Scenario:  Symptoms  SO  and  S 1  were  reported  on  the  AFTO  Form  349,  and  F2 
and  F3  were  designated  as  the  true  system  faults.  MOTs  have  more  than  one  test  outcome.  When 
more  than  one  outcome  is  observed,  does  the  diagnostic  module  have  the  capability  to  rectify  more 
than  one  outcome? 

3.  Steps: 

Action  Recommendation  Comment 


a.  Initialize  w/SO&  SI  T7 

b.  T7  (outcome  1.2  &  1.3)  R2 


c.  R2 

d.  Functional  check  (fail) 

e.  RO 

f.  Functional  check  (fail) 

g.  R4 


Functional  check 
RO 

Functional  check 
R4 

Repeat  steps  b.-g. 


Only  one  outcome  can  be 
entered  for  the  MOT. 

Outcome  1.2  was  chosen,  and 
the  fault  associated  with 
Outcome  1.3  is  exculpated. 


6.  Test  Result*  The  IMIS  diagnostic  module  does  not  have  the  ability  to  accept  more  than  one 
outcome  from  an  MOT.  Oily  one  outcome  can  be  chosen,  and  the  faults  associated  with  the 
remaining  outcomes  are  exculpated. 
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IMIS  Diagnostic  Module  V&V  Test  Sheet 


1.  Test  No.  _32 


2.  Test  Scenario: 


3.  Test  Model:  Testing  was  performed  to  investigate  the  diagnostic  module's  ability  to  FOM  when 
a  rectification  is  being  performed. 

4.  Test  Procedure/Scenario: 

Selection  Action 


a.  N/A  Initialize 

b.  Rectification  Perform  access  step 

c.  FOM  Remove 

d.  Rectification  Remove 

e.  Rectification  Install 

f.  FOM  Install 

5.  Steps: 

Action  Recommendation  Comment 


a.  Initialize  Select  rectification 

b.  Perform  access  step  Facilitate  other  maintenance  A  component  from 

another  rectification 
must  be  removed  prior 
to  performing  the 
diagnostic  module’s 
recommended  action 
from  step  a. 

c.  Select  TOC  TOC  deactivated,  use 

backup  function 
instead. 

d.  Backup  went  to  FC  Functional  check 

e.  Select  Back  Up  option  Failed  faults  found  Backup  passed  FC, 

diagnostics  end. 
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f.  Select  some  rectification  Diagnostics  were 

completed  with  step 
e.,  and  selection  of 
some  other 
maintenance  action 
could  not  be 
performed. 

6.  Test  Result:  The  IMIS  diagnostic  module  will  not  facilitate  other  maintenance  actions.  TO 
procedures  must  correctly  address  all  maintenance  activity  associated  with  the  removal  and 
replacement  of  components;  otherwise,  the  rectification  cannot  be  completed  successfully. 
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1.  Test  No.  _22 

2.  Test  Scenario:  Cannibalization 

3.  Test  Procedure/Scenario:  A  method  was  investigated  to  provide  the  proper  TO  sequencing  for 
cannibalization  of  another  A/C. 

4.  Test  Procedure/Scenario: 

Selection  Action  Aircraft 


a.  N/A 

b.  Rectification 

c.  Rectification 

d.  Rectification 

e.  Isolation 

f.  Rectification 

g.  Rectification 

h.  N/A 

i.  Rectification 

j.  Rectification 

5.  Steps: 

Action 

a.  Initialize 

b.  Perform  removal 
sequence 

c.  Select  TOC 

d. 

e. 

f. 


g- 


Initialize 

Remove 

Remove 

Install 

Functional  check  (pass) 

Remove 

Install 

Functional  check  (pass) 

Restore 

Restore 

Recommendation 
Select  rectification 


Continue  same  rectification 


A/C-l 

A/C-2 

A/C-l 

A/C-l 

A/C-l 

A/C-2 

A/C-2 

A/C-l 

A/C-2 

Comment 


Remove  Pan  1  (PI)  from 
A/C-l 

TOC  deactivated,  use  Back 
Up  instead. 


Remove  PI  from  A/C-2. 
There  is  no  facility  within  the 
diagnostic  module  to 
document  rectification 
procedures  are  being 
performed  on  two  A/C. 


Back  up  to  top  of  remove 

Select  same  rectification 

Perform  removal 
sequence 
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h.  Perform  replacement  Functional  check 
sequence 


Replace  PI  (from  A/C-2)  into 
A/C-l. 


i  Functional  check  Failed  faults  found 

(pass) 


j.  Select  TOC 


TOC  deactivated 


Diagnostics  end  and  the 
rectification/isolation  is 
complete  on  A/C-l,  but  A/C-2 
parts  are  lying  all  over  the 
ramp. 


If  PI  was  for  isolation, 
procedures  cannot  be  accessed 
to  take  PI  from  A/C-l  and 
restore  A/C-2.  Otherwise,  if 
cannibalization  was  for 
rectification,  same  comments 
as  step  g. 


S.  Test  Result:  The  IMIS  diagnostic  module  did  not  have  the  facility  or  the  ability  to  perform 
cannibalization  procedures.  In  die  event  that  the  cannibalization  effort  was  for  isolation, 
diagnostics  end  without  restoring  A/C-2.  If  the  effort  was  for  rectification,  A/C-2  parts  are  lying 
all  over  the  ramp. 
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1.  Test  No.  _24 

2.  Test  Scenario:  Nonfunctional  Kev  Entries 

3.  Scenario/Result:  When  performing  normal  diagnostic  sequences,  several  key  entries  are 
required  to  provide  the  diagnostic  module  with  information  about  the  selection  and  the  outcomes  of 
actions.  If  a  nonfunctional  key  entry  is  made,  processing  errors  occur  and  diagnostics  "bomb." 
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1.  Test  No.  35 

2.  Test  Scenario:  Recalculation  of  List  Options 

3.  Scenario/Result:  Best  tests,  best  rectifications,  and  best  actions  are  three  options  that  are  used 
extensively  to  view  and  select  ranked  actions.  Each  list  requires  an  enormous  amount  of 
calculation  which  is  performed  when  diagnostics  are  initialized.  In  the  process  of  performing 
validation  and  verification  on  the  large  data  base,  ranked  options  lists  were  selected  to  perform 
actions.  This  led  to  the  discovery  that  recalculations  and  rerankings  were  performed  each  time  an 
action  list  was  selected  for  display.  The  time  required  to  perform  this  was  approximately  15 
seconds  and  was  completely  unnecessary  since  the  calculations  and  rankings  were  performed 
during  the  initialization  of  the  diagnostic  module.  This  inefficiency  may  become  a  major  problem 
when  a  full  data  base  is  installed  and  several  symptoms  are  observed. 
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